Language Programs Ranked by Vocabulary Outcomes
One criticism I often see of meta-analysis is that different studies are measuring different outcomes. For example one reading program study might measure letter ID and phonemic awareness, whereas another might measure word ID and comprehension. Even more problematic, some studies list their impact without listing their measurement criteria. Moreover, the demographics of different studies are often very different, and different age demographics lead to different results. Earlier this year I put out a large, but non-peer reviewed meta-analysis of language programs. After doing so, I have often gotten the question, what is the best program for my specific situation. In order to address this question, and to help correct for the above listed (valid) criticism of meta-analysis, I have decided to put a small series of sub-analyses that break down my original results according to more specific outcomes, and rank programs based on their effect sizes, according to these outcomes.
This analysis provides more specific outcomes for people to look at. However, it also lowers the total sample size and statistical power of the analysis. This is especially true, as most studies only look at a small number of specific outcomes, with this in mind, readers should note that most language programs will be excluded from most sub-analyses, due to a lack of research. If you would like to learn more about the methodology behind this analysis, you can find the original sourced article here: https://www.teachingbyscience.com/a-meta-analysis-of-language-programs
On this page, you can find language programs ranked by effect size for vocabulary outcomes. Please note that this page will be updated over time, as I add new research.
Written by Nathaniel Hansford
Last Edited 2022-05-28
Lovett, M. W., Frijters, J. C., Wolf, M., Steinbach, K. A., Sevcik, R. A., & Morris, R. D. (2017). Early intervention for children at risk for reading disabilities: The impact of grade at intervention and individual differences on intervention outcomes. Journal of Educational Psychology, 109(7), 889-914. doi:http://dx.doi.org/10.1037/edu0000181
Amplify. (2019). Amplify CKLA AZ grade 5 efficacy research report. Retrieved from <https://amplify.com/wp-content/uploads/2019/12/CKLA_AZ-grade-5-efficacy-research-report.pdf>.
CKLA. (2019). CK Early Literacy Pilot. Retrieved from <https://amplify.com/wp-content/uploads/2019/12/CKLA-Early-Literacy-Pilot.pdf>.
Cabell, S.Q., White, T.G., Kim, J., Hwang, H., & Gale, C. (2019, December). Impact of the Core Knowledge Language Arts read-aloud program on kindergarteners’ vocabulary, listening comprehension, and general knowledge. Paper presented at the annual meeting of the Literacy Research Association, Tampa, FL.
J, Wedman. (2004). Core Knowledge Curriculum and School Performance. University of Missour. Retrieved from <https://www.coreknowledge.org/wp-content/uploads/2016/12/CK_National_Study_2004.pdf>.
Nasrawi, A., & Al-Jamal, D. (2017). The Effect of Using Jolly Phonics on Jordanian First Grade Pupils’ Reading. International Online Journal of Education & Teaching, 4(2), 106–119.
Callinan, C., & van der Zee, E. (2010). A comparative study of two methods of synthetic phonics instruction for learning how to read: Jolly Phonics and THRASS. Psychology of Education Review, 34(1), 21–31.
M, Stuart. (1999). Getting ready for reading: Early phoneme awareness and phonics teaching improves reading and spelling in inner-city second language learners. British Journal of Psychology. Retrieved from <https://jolly2.s3.amazonaws.com/Research/Getting%20Ready%20for%20Reading.pdf>.
C, Crane, Et, Al. (1999). Improving early language and literacy skills: differential effects of an oral language versus a phonology with reading intervention. University of York. Retrieved from <https://jolly2.s3.amazonaws.com/Research/BowyerCrane%20etal2007proof.pdf>.
L, Farokhbakht. The Effect of Using Multisensory-based Phonics in Teaching Literacy on EFL Young Female/Male Learners' Early Reading Motivation. University of Isfahan. Retrieved from <https://jolly2.s3.amazonaws.com/Research/The%20Effect%20of%20Using%20Multisensory-based%20Phonics%20in%20Teaching%20Literacy%20on%20EFL%20Young%20FemaleMale%20Learners'%20Early%20Reading%20Motivation.pdf>.
N, Katechaiyo, et al. EFFECTS OF JOLLY PHONICS INSTRUCTION FOR PUPIL BOOK 1 ON REDING ABILITY OF THAI EFL YOUNG LEARNERS. Retrieved from <https://jolly2.s3.amazonaws.com/Research/Revealing%20the%20secrets%20of%20remarkable%20improvement%20of%20Thai%20EFL%20young%20learners_Aug.2021.pdf>.
Republic of Gambia. (2009). IMPACT ASSESSMENT OF INTERVENTIONS ON EARLY GRADE READING ABILITY (EGRA) IN SCHOOLS. Retrieved from <https://www.jollylearning.co.uk/evidence/research/>.
Government of Nigeria. (2014). REPORT ON THE MONITORING EXERCISE FOR THE IMPLEMENTATION OF JOLLY PHONICS APPROACH IN THE FEDERAL CAPITAL TERRITORY, ABUJA. Retrieved from <https://s3.amazonaws.com/jolly2/Research/Jolly+Phonics+in+FCT.pdf>.
N, Katechaiyo, Et al. EFFECTS OF JOLLY PHONICS INSTRUCTION FOR PUPIL BOOK 1 ON READING ABILITY OF THAI EFL YOUNG LEARNERS. Retrieved from <https://jolly2.s3.amazonaws.com/Research/Revealing%20the%20secrets%20of%20remarkable%20improvement%20of%20Thai%20EFL%20young%20learners_Aug.2021.pdf>.
Skindrud, K., & Gersten, R. (2006). An Evaluation of Two Contrasting Approaches for Improving Reading Achievement in a Large Urban District. Elementary School Journal, 106(5), 389–407. https://doi-org.ezproxy.lakeheadu.ca/10.1086/505437
Stockard, J. (2010). Promoting Reading Achievement and Countering the “Fourth-Grade Slump”: The Impact of Direct Instruction on Reading Achievement in Fifth Grade. Journal of Education for Students Placed at Risk, 15(3), 218–240. https://doi.org/10.1080/10824669.2010.495687
Vaden-Kiernan, M., Borman, G., Caverly, S., Bell, N., Sullivan, K., Ruiz de Castilla, V., Fleming, G., Rodriguez, D., Henry, C., Long, T., & Hughes Jones, D. (2018). Findings From a Multiyear Scale-Up Effectiveness Trial of Open Court Reading. Journal of Research on Educational Effectiveness, 11(1), 109–132. https://doi-org.ezproxy.lakeheadu.ca/10.1080/19345747.2017.1342886
Borman, G. D., Dowling, N. M., & Schneck, C. (2008). A Multisite Cluster Randomized Field Trial of Open Court Reading. Educational Evaluation and Policy Analysis, 30(4), 389–407. https://doi.org/10.3102/0162373708326283
Rachel Schechter, Paul Macaruso, Elizabeth R. Kazakoff & Elizabeth Brooke (2015) Exploration of a Blended Learning Approach to Reading Instruction for Low SES Students in Early Elementary Grades, Computers in the Schools, 32:3-4, 183-200, DOI: 10.1080/07380569.2015.1100652
Macaruso, P., Hook, P. E., & McCabe, R. (2006). The efficacy of computer-based supplementary phonics programs for advancing reading skills in at-risk elementary students. Journal of Research in Reading, 29(2), 162–172. https://doi-org.ezproxy.lakeheadu.ca/10.1111/j.1467-9817.2006.00282.x
Macaruso, P., & Rodman, A. (2011). Efficacy of Computer-Assisted Instruction for the Development of Early Literacy Skills in Young Children. Reading Psychology, 32(2), 172–196. https://doi-org.ezproxy.lakeheadu.ca/10.1080/02702711003608071
Hurwitz, L.B. (2020). Supporting Struggling and Non-Proficient Middle School Readers with the Lexia PowerUp Literacy Program. Concord, MA: Lexia Learning Systems LLC, A Rosetta Stone Company
S, Wilkes, Et al. (2016). Exploration of a Blended Lerning Approach to Reading Instruction in Second Grade. Edmedia. Retrieved from <https://www.lexialearning.com/user_area/content_media/raw/EdMediaPresentation_TitleI.pdf>.
Macaruso, P., & Walker, A. (2008). The Efficacy of Computer-Assisted Instruction for Advancing Literacy Skills in Kindergarten Children. Reading Psychology, 29(3), 266–287. https://doi-org.ezproxy.lakeheadu.ca/10.1080/02702710801982019
Schechter, R., Macaruso, P., Kazakoff, E.R. and Brooke, E. (2015). Exploration of a blended learning approach to reading instruction for low SES students in early elementary grades. Computers in the Schools, 32, 183–200.
Macaruso, P., & Rodman, A. (2011). Benefits of Computer-Assisted Instruction to Support Reading Acquisition in English Language Learners. Bilingual Research Journal, 34(3), 301–315. https://doi-org.ezproxy.lakeheadu.ca/10.1080/15235882.2011.622829
Paul Macaruso & Alyson Rodman (2009) Benefits of computer‐assisted instruction for struggling readers in middle school, European Journal of Special Needs Education, 24:1, 103-113, DOI: 10.1080/0885625080259677
C, Schieffer. (2002). An Analysis of the Reading Mastery Program: Effective Components and Research Review. Journal of Direct Instruction. Volume 2: Issue 2. Retrieved from <https://www.nifdi.org/research/journal-of-di/volume-2-no-2-summer-2002/442-an-analysis-of-the-reading-mastery-program-effective-components-and-research-review/file.html>.
Yu, L., & Rachor, R. (2000, April). The two-year evaluation of the three-year Direct Instruction program, in an urban public school system. Presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
O'Connor, R. E., Jenkins, J. R., Cole, K. N., & Mills, P. E. (1993). Two approaches to reading instruction with children with disabilities: does program design make a difference?. Exceptional children, 59(4), 312–323. https://doi.org/10.1177/001440299305900404
The Development of Early Academic Success: The Impact of Direct Instruction’s Reading Mastery. (2010). Journal of Behavior Assessment & Intervention in Children, 1(1), 2–24. https://doi-org.ezproxy.lakeheadu.ca/10.1037/h0100357
R, Colvin. Reading Recovery Revisited. The School Superintendent Association. Retrieved from <https://aasa.org/SchoolAdministratorArticle.aspx?id=15712> .
D’Agostino, J. V., & Harmey, S. J. (2016). An International Meta-Analysis of Reading Recovery. Journal of Education for Students Placed at Risk (JESPAR), 21(1), 29–46. https://doi.org/10.1080/10824669.2015.1112746
Gardner, J., Sutherland, A., & Meenan-Strain, C. (1998) Reading Recovery in Northern Ireland: The first two years. Belfast, Ireland: Blackstaff.
Schwartz, Robert. (2005). Literacy Learning of At-Risk First-Grade Students in the Reading Recovery Early Intervention.. Journal of Educational Psychology. 97. 257-267. 10.1037/0022-0622.214.171.1247.
Sirinides, P., Gray, A., & May, H. (2018). The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation. Educational Evaluation and Policy Analysis, 40(3), 316–335. https://doi.org/10.3102/0162373718764828
Burroughs-Lange, S. (2008). Comparison of literacy progress of young children in London Schools: A RR Follow-Up Study.
London, UK: Institute of Education. Retrieved from https://www.ioe.ac.uk/Comparison_of_Literacy_Progress_of_Young_Children_in_London_Schools_-_A_Reading_Recovery_Follow_up_Study_.pdf
Hurry, J., & Sylva, K. (2007). Long-term outcomes of early reading intervention. Journal of Research in Reading, 30(3), 227–248. https://doi.org/10.1111/j.1467-9817.2007.00338.x
Pinnell, Html & Lyons, Carol & Deford, Diane & Bryk, Anthony & Seltzer, Michael. (1994). Comparing Instructional Models for the Literacy Education of High-Risk First Graders.
Reading Research Quarterly. 29. 10.2307/747736.
Holliman, A.J., and Hurry, J. (2013) The effects of Reading Recovery on children's literacy progress and Special Educational Needs status: A three-year follow-up study. Educational Psychology, 33(6), pp. 719-733
Shanahan, T., & Barr, R. (1995). Reading Recovery: an independent evaluation of the effects of an early instructional intervention for at-risk learners. Reading Research Quarterly, 30, 958–996. https://doi-org.ezproxy.lakeheadu.ca/10.2307/748206
Lyons, C. A. (1988). Reading Recovery: Early intervention for at-risk first graders (Educational
Research Service Monograph). Arlington, VA: Educational Research Service. (ERIC Document Reproduction Service No. ED303790).
DeFord, D., Pinnell, G. S., Lyons, C. A., & Young, P. (1987). Reading Recovery program: Report of the follow-up studies (Vol. VII). Columbus, OH: The Ohio State University.