Evidence Based Education
Is Fountas and Pinnell's LLI Evidence Based?
Fountas and Pinnell intervention program Leveled Literacy Instruction is a popular balanced literacy program. This program focuses on the use of fluency instruction via leveled texts, cueing, and guided reading. LLI does include some phonics, but it is limited and embedded within their fluency instruction. Leveled Literacy Instruction was inspired by Marie Clay’s Reading Recovery program. Recently, LLI has faced intense scrutiny for its efficacy. However, much of the criticism and support of LLI has been qualitative in nature. For example, the Fountas and Pinnell website, has a bibliography of studies on LLI and cites that this research has been positive, as proof of their efficacy. However, followers of this blog know that we believe efficacy (in education) can only be determined via quantitative measures and meta-analysis and the Fountas and Pinnell website does not list any quantitative information, in support of their programs efficacy.
I looked for a meta-analysis of the topic; however, I could not find any current meta-analysis of the topic, despite the fact that Fountas and Pinnell’s program is the most widely used English Language program in the world and despite the fact that it has faced intense scrutiny. Evidence for ESSA, as evaluated by LLI; however, they only looked at 2 studies and found a mean effect size of .13, which is statistically negligible. For the purposes of this blog, we conducted a meta-analysis of LLI, to evaluate the efficacy of the program. In order to conduct this analysis, a systematic search was conducted for LLI studies on the LLI website, on google, Education Source, and Sage Journal. All studies that did not have a control group and/or enough statistical detail to calculate an effect size were excluded. Next, effect sizes were calculated using the Cohen's d formula (for studies with sample sizes above 50) and Hedge's g (for studies with sample sizes below 50.
Results:
The systematic search found 7 studies on the topic of LLI instruction. However, no studies were found on Fountas and Pinnell Core instruction. 5 Studies were conducted by the Center for Research Policy in Education. 2 Studies were conducted by independent authors.
.png)
On average, efficacy studies of LLI had an ES of .31, which is in the low range. Meaning that LLI does have a positive impact, but that impact is below the average impact of a literacy intervention and below the average impact of many other literacy programs. According to the NRP meta-analysis phonics has a mean effect size of .41 and in my own meta-analysis of the topic phonics has a mean effect size of .44. This suggests that on average phonics instruction produces higher outcomes than LLI.
I do have some concerns about the LLI research. A large number of the LLI studies were done by the same institute, the University of Memphis Center for Research Policy in Education (CRPE). Indeed, the LLI website only cites one study not done by this institute. These studies, as far as I could find, all showed effect sizes much more positive than all other studies done on the topic. Indeed one CRPE study found a mean ES of .44, which was an outlier compared to the rest of studies on the topic. The CRPE, also stated in some of their studies that effects were not statistically significant for some measurements, but did not provide the corresponding data. This methodology means that only the statistically significant/positive effects were reported in the CRPE studies, which inflates their mean results.
I also found some strange statistical anomalies in the CRPE papers. For example, the 2015 paper by the institute, had 3 effect sizes based on identical standard deviations and results for both their control group, intervention group, and on their pre and post tests. The resulting effect sizes were also extremely high. For this reason, I excluded these 3 effect sizes from my calculations, as suspicious. Comparatively, the two papers done independent of the CRPE on the topic both showed statistically insignificant to negative research outcomes for LLI.
Final Grade: C+: The program is not research based, IE most of the principles are not supported by the meta-analysis data. A mean effect size of .31 was found.
Qualitative Grade: 3/10
The program includes the following evidence based instructional types: Fluency, comprehension, and spelling.
Written by Nate Joseph and Joshua King
Last Edited 2023-01-17
Disclaimer: The following research has not yet been peer reviewed.
References:
Center for Research Policy in Education. (2012). Efficacy of the Leveled Literacy Intervention System for K–2 Urban Students: An Empirical Evaluation of LLI in Denver Public Schools Study Dates: 2011–2012. University of Memphis. Retrieved from <https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_CREP-LLI-Efficacy-Full-Report-2012.pdf>.
Center for Research Policy in Education. (2010). Implementation of Effective Intervention: An Empirical Study to Evaluate the Efficacy of Fountas & Pinnell’s Leveled Literacy Intervention System (LLI) Retrieved from <https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_CREP-LLI-Efficacy-Full-Report-2010.pdf>.
Center for Research Policy in Education. (2016). The Efficacy of the Leveled Literacy Intervention System for Students in Grades 3–5: Data Summary Report for Abilene Independent School District 2015–2016. University of Memphis. Retrieved from <https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-AISD_FULL_REPORT-web.pdf>.
Center for Research Policy in Education. (2015). The Efficacy of the Leveled Literacy Intervention System for Students in Grades 3–5: Data Summary Report for Denver Public Schools 2015–2016. University of Memphis. Retrieved from <https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-DPS_FULL_REPORT-web.pdf>.
Center for Research Policy in Education. (2016). The Efficacy of the Leveled Literacy
Intervention System for Students in Grades 3–5: Data Summary Report for Sandwich Public Schools 2015–2016. University of Memphis. Retrieved from <https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-SPS_FULL_REPORT-web.pdf>.
Heinmann. (2010). Levelled Literacy Intervention, Research and Data Collection Project. Fountas and Pinnell. Retrieved from <https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_Research-and-Data-Collection-Project-Report.pdf>.
Gonzalez, et al. (2018). Challenges in Adolescent Reading Intervention: Evidence from a Randomized Control Trial. Mathmatica Policy Research. Retrieved from <https://files.eric.ed.gov/fulltext/ED587404.pdf>.
Taylor, Lisa, "The Effects of Leveled Literacy Intervention for Students in the RtI Process" (2017). Walden Dissertations and Doctoral Studies. 3960.
Lang, Laura & Torgesen, Joseph & Vogel, William & Chanter, Carol & Lefsky, Evan & Petscher, Yaacov. (2009). Exploring the Relative Effectiveness of Reading Interventions for High School Students. Journal of Research on Educational Effectiveness. 2. 149-175. 10.1080/19345740802641535.
Metz, Tracie Jean, "A case study: Effects of using leveled literacy intervention on fourth and fifth grade students' reading achievement" (2014). ETD Collection for Fayetteville State University. AAI3581423.
https://digitalcommons.uncfsu.edu/dissertations/AAI3581423
J, Majewski. (2018). The effects of a leveled literacy inter acy intervention (LLI) on elementary age students reading below grade level. Rowan University. Retrieved from <https://rdw.rowan.edu/cgi/viewcontent.cgi?article=3574&context=etd>.
J, Hattie. (2021). Visible Learning Metax. Retrieved from <www.visiblelearningmetax.com>.
N, Hansford. (2021). Morphology. Pedagogy Non Grata. Retrieved from <https://nathanielhansford.wixsite.com/website/morphology>.
Linnea, et al. (2001). Systematic Phonics Instruction Helps Students Learn to Read: Evidence From the National Reading Panel’s Meta-Analysis. Ontario Institute for Studies in Education. Retrieved from <https://www.dyslexie.lu/JDI_02_02_04.pdf>.