Fountas and Pinnell
Fountas and Pinnell offers both an intervention program called Leveled Literacy Instruction and a core instruction program called Fountas & Pinnell Classroom. The program is based on the principles of balanced literacy instruction and is possibly the most popular reading program in the world. This program focuses on the use of fluency instruction via leveled texts, cueing, and guided reading. Fountas and Pinnell instruction does include some phonics, but it is typically limited and embedded within their fluency instruction. Leveled Literacy Instruction was inspired by Marie Clay’s Reading Recovery program. Recently, Fountas and Pinnell has faced intense scrutiny for its efficacy. However, much of the criticism and support of Fountas and Pinnell has been qualitative in nature. For example, the Fountas and Pinnell website, has a bibliography of studies on LLI and cites that this research has been positive, as proof of their efficacy. However, followers of this blog know that we believe efficacy (in education) can only be determined via quantitative measures and meta-analysis and the Fountas and Pinnell website does not list any quantitative information, in support of their programs efficacy.
We looked for a meta-analysis of the topic; however, we could not find any current meta-analysis of the topic, despite the fact that Fountas and Pinnell’s program is possibly the most widely used English Language program in the world and despite the fact that it has faced intense scrutiny. Evidence for ESSA has evaluated LLI; however, they only looked at 2 studies and found a mean effect size of .13, which is statistically negligible. For the purposes of this blog, we reviewed studies of Fountas and Pinnell, to evaluate the efficacy of the program.
In order to conduct this analysis, a systematic search was conducted for Fountas & Pinnell studies on the company website, on google, Education Source, and Sage Journal. We searched the term “Fountas and Pinnell” in the academic data-bases to find studies. In Google we searched the term “Fountas and Pinnel Research.” All studies that did not have a control group and/or enough statistical detail to calculate an effect size were excluded. In total our search turned up 355 papers. However, the majority of papers were not relevant. Most often papers were excluded for either not being experimental or for duplication. At the end of our initial search were located 7 relevant studies that were either experimental or quasi-experimental. Next, effect sizes were calculated using the Cohen's d formula. All effect sizes were first calculated by the primary author and then independently re-calculated by either the second or third author.
The systematic search found 7 relevant studies on the topic of LLI instruction. However, no studies were found on Fountas and Pinnell Core instruction. 5 Studies were conducted by the Center for Research Policy in Education. 2 Studies were conducted by independent authors. **Update** After this original analysis and article was published, Pedagogy Non Grata helped to conduct two small quasi-experimental studies that were relevant to his topic. Both studies looked at grade 1 classes in which the control group and treatment groups each received instruction based on F and P materials. However, the treatment group also received additional phonics instruction, via the use of mnemonics. These studies had three serious limitations: they have not been peer-reviewed, they were conducted retroactively (which runs the risk of bias), and they were sponsored studies (which also runs the risk of bias). However, we believe these studies are important to consider for two reasons:
To the best of our knowledge these studies are the only experimental studies ever conducted which compared Fountas and Pinnell materials to a comparison group that received additional phonics.
To the best of our knowledge these studies are the only experimental studies ever conducted which examined Fountas and Pinnel core instruction.
In order to limit the potential bias for these studies, we had all parties sign strict ethics agreements that stated all data had to be shared in good faith, all results had to be published, and that all parties must act in the best interest of scientific accuracy. We also had an outside organization independently review our research (LXD research).
Table 1: Studies Included
On average, we found a mean effect size for LLI of ES of .10, (95% CI= [-.26, .46]) which is positive, but negligible according to Cohen’s guide. However, most of these studies were comparing LLI to business as usual control groups. The studies which compared LLI to business as usual control groups on average showed a mean effect size of .31, which is statistically significant, but small. If we look at the studies that compared Fountas and Pinnell to additional phonics, we saw a mean effect size of -.63. Comparatively, according to the NRP meta-analysis phonics has a mean effect size of .41 and in our own meta-analysis of the topic (unpublished and submitted for peer review) phonics has a mean effect size of .44. This suggests that on average systematic phonics instruction programs produce higher outcomes than LLI and Fountas & Pinnell Classroom.
We do have some concerns about the LLI research. A large number of the LLI studies were done by the same institute, the University of Memphis Center for Research Policy in Education (CRPE). Indeed, the LLI website only cites one study not done by this institute. These studies, as far as we could find, all showed effect sizes much more positive than all other studies done on the topic. The average effect size for CRPE studies was .44. Whereas the average effect size for studies not conducted by the CRPE was -.32 and all studies not conducted by the CRPE showed either negligible or negative results. The CRPE, also stated in some of their studies that effects were not statistically significant for some measurements, but did not provide the corresponding data. This methodology means that only the statistically significant/positive effects were reported in the CRPE studies, which inflates their mean results.
We also found some strange statistical anomalies in the CRPE papers. For example, the 2015 paper by the institute, had 3 effect sizes based on identical standard deviations and results for both their control group, intervention group, and on their pre and post tests. The resulting effect sizes were also extremely high (>3.00). For this reason, we excluded these 3 effect sizes from this paper, as outliers.
On average, studies with smaller sample sizes are more prone to bias. To examine whether or not the results of this analysis were biased by small sample sizes, we conducted a funnel plot analysis, as can be seen below.
Figure 1: Fountas and Pinnell Funnel Plot Analysis
This funnel plot does seem to suggest that a large number of studies were outliers. Moreover, it suggests that 2 of the highest effect sizes were from small sample studies and that 2 of the lowest effect sizes were from low sample size studies.
Overall this research seems to suggest that Fountas and Pinnel show below average results, when compared to approaches that include more systematic phonics. However, these results also suggest that Fountas and Pinnell instructional materials can be improved, simply by adding more phonics instruction.
Final Grade: C-: The program is not research based, IE most of the principles are not supported by the meta-analysis data. The mean effect size found was negligible.
Qualitative Grade: 3/10
The program includes the following evidence based instructional types: Fluency, comprehension, and spelling.
Written by Nate Joseph, Joshua King, and Sky McGlynn
Last Edited 2022-12-11
Disclaimer: The following research has not been peer reviewed.
Center for Research Policy in Education. (2012). Efficacy of the Leveled Literacy Intervention System for K–2 Urban Students: An Empirical Evaluation of LLI in Denver Public Schools Study Dates: 2011–2012. University of Memphis. https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_CREP-LLI-Efficacy-Full-Report-2012.pdf
Center for Research Policy in Education. (2010). Implementation of Effective Intervention: An Empirical Study to Evaluate the Efficacy of Fountas & Pinnell’s Leveled Literacy Intervention System (LLI). University of Memphis. https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_CREP-LLI-Efficacy-Full-Report-2010.pdf
Center for Research Policy in Education. (2016). The Efficacy of the Leveled Literacy Intervention System for Students in Grades 3–5: Data Summary Report for Abilene Independent School District 2015–2016. University of Memphis. https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-AISD_FULL_REPORT-web.pdf
Center for Research Policy in Education. (2015). The Efficacy of the Leveled Literacy Intervention System for Students in Grades 3–5: Data Summary Report for Denver Public Schools 2015–2016. University of Memphis. https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-DPS_FULL_REPORT-web.pdf.
Center for Research Policy in Education. (2016). The Efficacy of the Leveled Literacy
Intervention System for Students in Grades 3–5: Data Summary Report for Sandwich Public Schools 2015–2016. University of Memphis. https://www.fountasandpinnell.com/shared/resources/LLI%203-5%202015_16-SPS_FULL_REPORT-web.pdf
Heinmann. (2010). Leveled Literacy Intervention, Research and Data Collection Project. Fountas and Pinnell. https://www.fountasandpinnell.com/shared/resources/FP_LLI_Research_Research-and-Data-Collection-Project-Report.pdf
Gonzalez, et al. (2018). Challenges in Adolescent Reading Intervention: Evidence from a Randomized Control Trial. Mathematical Policy Research. https://files.eric.ed.gov/fulltext/ED587404.pdf
Taylor, Lisa, "The Effects of Leveled Literacy Intervention for Students in the RtI Process" (2017). Walden Dissertations and Doctoral Studies. 3960.
Lang, Laura & Torgesen, Joseph & Vogel, William & Chanter, Carol & Lefsky, Evan & Petscher, Yaacov. (2009). Exploring the Relative Effectiveness of Reading Interventions for High School Students. Journal of Research on Educational Effectiveness. 2. 149-175. 10.1080/19345740802641535.
Metz, Tracie Jean, "A case study: Effects of using leveled literacy intervention on fourth and fifth grade students' reading achievement" (2014). ETD Collection for Fayetteville State University. AAI3581423.
J, Majewski. (2018). The effects of a leveled literacy literacy intervention (LLI) on elementary age students reading below grade level. Rowan University. https://rdw.rowan.edu/cgi/viewcontent.cgi?article=3574&context=etd
Hattie, J. (2021). Visible Learning Metax. www.visiblelearningmetax.com
N, Hansford, Joshua King & Sky McGlynn. (2023). Secret Stories Case Study: Michigan, Grade 1. https://www.canva.com/design/DAFhVeanjv0/Re142oB6XUD_15TIPPwExA/view?utm_content=DAFhVeanjv0&utm_campaign=designshare&utm_medium=link&utm_source=homepage_design_menu
N, Hansford, Joshua King & Sky McGlynn. (2023).Secret Stories RCT Study. First Grade Arkansa. https://www.canva.com/design/DAFhVTaFLlU/X971hgF8gysi_TRorU5XPg/view?utm_content=DAFhVTaFLlU&utm_campaign=designshare&utm_medium=link&utm_source=homepage_design_menu