Evidence Based Education
Commonly Used Math Pedagogies and Factors Ranked by Effect Sizes Found In Meta-Analysis
Research results can be a fickle thing. I have seen phonics studies with negative effect sizes and Balanced Literacy studies with very high effect sizes. But if we look across the breadth of the literature, we see a clear trend in that phonics outperforms balanced literacy. For this reason meta-analysis is clearly necessary. This allows us to answer the question, what does the majority of the scientific literature say on a specific topic. Of course this is problematic also. Different studies are done across different time horizons, with different demographics, with different tests, by different researchers.
While, meta-analysis is supposed to remove the variability by ultimately increasing the total sample, some of the variability is undoubtedly context specific, and not random. For example, multi-step word problems show very low results in elementary school, but very high results in secondary school. The more we combine and generalize studies, the more we risk taking them out of a context, in which they mean something.
That being said, the variability we see within studies is far too great, within the field of education, for me to believe it is reasonable to accept the results of any individual study in isolation. Personally, I believe the answer for this is meta-analysis that includes as much context specific data, as possible, such as age, subject, and condition. Which is why my website, as of late, has gone through contextual meta-analyses on a variety of topics. However, I also think we have to realize that teaching is a time constricted activity. Teachers only have limited resources and time, so they need to pick strategies/factors which give them the most yield for their time. However, to determine what strategies/factors can best help them reach success in their classrooms, they need to be able to compare the effects of different pedagogies, strategies, and factors. The only way to do this on a high level is secondary meta-analysis.
A secondary meta-analysis is a meta-analysis of meta-analyses and it comes with all the problems of a regular meta-analysis and then some. A common criticism can be that when you use a secondary meta-analysis you are “comparing apples to oranges”. And of course, this is at least partially true, as the student populations, ages, measures, calculations, etc, all end up being variable. Which means, the contextual specificity of the research is very limited. That being said, there is no more efficient methodology for finding the relative effectiveness of a pedagogy compared to multiple other pedagogies.
This article is an excerpt from a book I am currently writing. For it, I have created a secondary meta-analysis out of the 41 math meta-analyses and research studies I have reviewed over the last two years. Moreover, I have tried to correct for some of the contextual issues, by creating multiple other secondary meta-analyses that are context specific, which will follow up with in future articles.
Factors Glossary:
Ability Grouping: Grouping students according to their ability levels. Source: Secondary meta-analysis by author, 2022. Link for more information: https://www.teachingbyscience.com/differentiation
Accuracy Focused Instruction: Fluency instruction focused accuracy, over speed. Source: Methe 2012 meta-analysis. Link for more information: https://www.teachingbyscience.com/math-fact-fluency
Adaptive Process Tech: Technology that individualizes student instruction via AI. Source: Ran 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/technology-and-math
Calculators: The use of calculators on math achievement. Source: Ellington 2003 meta-analysis. Link for more information:
https://www.teachingbyscience.com/calculator
Cognitive Based Interventions: Includes meta-cognition strategies, schema instruction, self talk, and behavior management strategies. Source: Myers 2021 meta-analysis. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Collaborative Tech: Software that helps staff and students collaborate/communicate more easily, such as Google Classroom. Source: Ran 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/technology-and-math
Constructivist Games: Games based on constructivist math philosophy. Constructivist math philosophy dictates that “Learners are actively engaged in their own learning such that knowledge is assumed to be constructed by learners rather than transmitted. Constructivism closely relates to experiential and discovery learning. However, it adds the construction of personal meaning by the learner as a final step.” Source: Kacmaz 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/game-based-learning-in-math
Contingent Reinforcement: Giving specific rewards to students for specific math achievements, such as candy, for demonstrating mastery of a learning target. *This research was only on primary students, which is important as rewards research shows negative outcomes for older students. Source: Getson 2009 Meta-analysis. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Cooperative Classroom: A classroom that emphasizes the use of cooperation over competition. Source: Zhining 1995 meta-analysis.
Cover Copy Compare: Have the student copy down a set of math facts. Rewrite the math facts from memory. Then check their work. Source: Methe 2012 meta-analysis. Link for more information:
https://www.teachingbyscience.com/math-fact-fluency
CRA: Concrete-representational-Abstract, also referred to as graduated instruction. A lesson/unit planning method, in which teachers use manipulatives first, diagrams second, and abstract work last. Source: Methe 2012 meta-analysis. Link for more information:
https://www.teachingbyscience.com/math-fact-fluency
DI/Conceptual Games: Games that use direct instruction to teach math concepts. Source: Kacmaz 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/game-based-learning-in-math
Experiential Games: Games that help students learn math through experiential learning. Source: Kacmaz 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/game-based-learning-in-math
Experiential Games: Games that help students learn math concepts through experiential learning. Source: Kacmaz 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/game-based-learning-in-math
Feedback: Telling students how they can improve their work. Source: Getson 2009 meta-analysis. Link for more information: https://www.teachingbyscience.com/learning-disabilities-and-math
Feedback with Goal Setting: Giving students feedback and encouraging them to make goals based on that feedback. Source: Getson 2009 meta-analysis. Link for more information: https://www.teachingbyscience.com/learning-disabilities-and-math
Feedback with Optional Targeted Instruction: Giving students feedback and offering them additional targeted instruction, to help their individual needs. Source: Getson 2009 meta-analysis. Link for more information: https://www.teachingbyscience.com/learning-disabilities-and-math
Fluency Focused Instruction: Fluency instruction that prioritizes speed over accuracy. Myers 2021 meta-analysis. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Game Based Instruction: Teaching math via games. Source: Kacmaz 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/game-based-learning-in-math
Homework: Giving students math homework. Source: 2018 Fan meta-analysis. Link for more information: https://www.teachingbyscience.com/math-homework
Iterative Teaching: Teaching conceptual and procedural knowledge at the same time. Source: Durkin 2011 meta-analysis. Link for more information: https://www.teachingbyscience.com/conceptual-math
Kumon: Japanese math program that focuses on mastery instruction, fluency and individualization. Source: Un-peer reviewed meta-analysis by author. Link for more information: https://www.teachingbyscience.com/kumon
Learning Management Systems: Computer AI/softwares that provide virtual classrooms which connect with multiple other learning programs. Source: Sayjili 2021 meta-analysis. Link for more information: https://www.teachingbyscience.com/learning-management-systems
Manipulatives: Physical objects used to help teach students math. Source: Carbonneau 2013 meta-analysis. Link for more information: https://www.teachingbyscience.com/manipulatives
Mastery Teaching: Teaching content until you are sure the students completely understand it. Source: John Hattie secondary meta-analysis. Link for more information: https://www.visiblelearningmetax.com/Influences
Math Facts Instruction: Directly teaching arithmetics facts through memorization, such as the times-tables. Source: Cason 2019 meta-analysis. Link for more information: https://www.teachingbyscience.com/math-fluency
Montessori Style Games: Games made with the private school Montessori brand of inquiry based learning. Source: Kacmaz 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/game-based-learning-in-math
Multi-Step Word Problems: Word problems with multiple steps to solve. Source: Myers 2022 meta-analysis. Link for more information: https://www.teachingbyscience.com/word-problems
Number Sense Games: Games that are designed to increase students’ understanding of number sense. Source: Kacmaz 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/game-based-learning-in-math
Number Sense Instruction: Instruction designed to increase students' understanding of place value, arithmetic, as well as their overall math fluency. Source: Cason 2019 meta-analysis. Link for more information:
https://www.teachingbyscience.com/math-fluency
Number Sense Word Problems: Word problems regarding the domain of number sense. Source: Myers 2022 meta-analysis. Link for more information: https://www.teachingbyscience.com/word-problems
Number Talks: A branded form of math discussion that focuses on teaching students multiple heuristics, via a constructivist approach. Source: A literature review by author. Link for more information:
https://www.teachingbyscience.com/number-talks
Numeracy Instruction: “Numeracy skills may include mathematical-logical thinking, relational reasoning, and specific concepts foundational for number sense such as one-to-one correspondence.” Source: Cason 2019 meta-analysis. Link for more information:
https://www.teachingbyscience.com/math-fluency
Peer Tutoring: Having students help teach other students. Source: Getson 2009 meta-analysis. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Problem Based Learning: Teaching math via complex problems that students solve in groups. Source: Haas 2005. Link for more information: https://www.visiblelearningmetax.com/influences/view/problem-based_learning
Problem Solving Games: Games based on the principles of problem based learning. Source: Kacmaz 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/game-based-learning-in-math
Problem Solving Technology: Technology based games that are based on the principles of problem based learning. Source: Ran 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/technology-and-math
Scaffolded Examples: Providing students with an easy, moderate, and difficult math exemplar. Source: Getson 2009 meta-analysis. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Shema Instruction: Providing students with meta-cognition strategies for solving complex math problems, usually associated with word problems and situational problems. IE: Ask yourself, what is the question asking, what do you know, what don’t you know. Source: Myers 2021 meta-analysis. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Single Step Word Problems: Word problems that only require one operation to solve.
Source: Myers 2022 meta-analysis. Link for more information: https://www.teachingbyscience.com/word-problems
Speed Based Interventions: Skill and drill activities rapidly done, such as around the world, or flash cards. Source: Getson 2009 meta-analysis. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Student Verbalization: Students using self-talk, while trying to solve problems. Source: Getson 2009. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Technology: Use of technology for the purposes of math instruction. Source: Ran 2022 meta-analysis. Link for more information:
https://www.teachingbyscience.com/technology-and-math
Use of Multiple Heuristics: Teaching multiple procedures for each type of math problem. Source: Getson 2009. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Visual Representatives: Using diagrams to teach math. Source: Getson 2009. Link for more information:
https://www.teachingbyscience.com/learning-disabilities-and-math
Word Problems: Concrete math problems. Source: Myers 2022 meta-analysis. Link for more information: https://www.teachingbyscience.com/word-problems
Written by Nathaniel Hansford
Last Edited 2022-04-07
References:
Bethany Rittle-Johnson, and Michael Schneider. (2011). Developing Conceptual and Procedural Knowledge of Mathematics. Oxford Press. Page 9.
Rittle, et al. (2007). Common and Flexible Use of Mathematical Non Routine Problem Solving Strategies. Science and Education.
Cowan, R., Donlan, C., Shepherd, D.-L., Cole-Fletcher, R., Saxton, M., & Hurry, J. (2011). Basic calculation proficiency and mathematics achievement in elementary school children. Journal of Educational Psychology, 103, 786–803. doi: 10.1037/a0024556.
Schwartz, D. L., Chase, C. C., Chin, D. B., & Oppezzo, M. (2011). Practicing versus inventing with contrasting cases: the effects of telling first on learning and transfer. Journal of
Educational Psychology, 103, 759–775. doi: 10.1037/a0025140
Hiebert, J. & Grouws, D. (2009). Which teaching methods are most effective for maths? Better: Evidence-Based Education, 2, 10–11.
J, Hatie. (2017). Hattie Ranking: 252 Influences And Effect Sizes Related To Student Achievement. Visible Learning. Corwin. Retrieved from <https://visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/>.
McKenna, M. A., Hollingsworth, P. L., & Barnes, L. L. B. (2005). Developing Latent Mathematics Abilities in Economically Disadvantaged Students. Roeper Review, 27(4), 222–227. https://doi-org.ezproxy.lakeheadu.ca/10.1080/02783190509554322
Begum, et al. (2020). Effectiveness of Kumon Teaching Method for Academic Achievement of Children in Mathematics. Pakistan Journal of Education. Retrieved from <https://pdfs.semanticscholar.org/3ba0/d3b4a989aa3e30dedcb6dc2d5bf94121fa5f.pdf>.
S, Medina. (1989). Study of the Effects of the Kumon MethodUpon the Mathematical Development of a Group of Inner-City Junior High School Students. Retrieved from <https://files.eric.ed.gov/fulltext/ED331700.pdf>.
Usmadi, et al. (2020). The Effect of Application Kumon Learning Method in Learning Mathematics of Ability Troubleshooting Mathematics of Students. Journal of Physics. Retrieved from <https://iopscience.iop.org/article/10.1088/1742-6596/1429/1/012005/pdf>.
Sulasteri, et al. (2020). The effect of Kumon learning model on mathematics learning outcomes in cognitive style view. Journal of Physics.
NELSON, G.; MCMASTER, K. L. The Effects of Early Numeracy Interventions for Students in Preschool and Early Elementary: A Meta-Analysis. Journal of Educational Psychology, [s. l.], v. 111, n. 6, p. 1001–1022, 2019. DOI 10.1037/edu0000334. Disponível em: http://search.ebscohost.com.ezproxy.lakeheadu.ca/login.aspx?direct=true&db=eue&AN=137730895&site=ehost-live. Acesso em: 25 jul. 2020.
S, Parish. (2014). Number Talks. Math Solutions.
J, Hattie. (2018). Hattie Ranking: 252 Influences And Effect Sizes Related To Student Achievement Visible Learning. Retrieved from <https://visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/>.
A, Stewarts. (2016). The Impact of Daily Number Talks on the Development of Mental Math Abilities of Second Graders within a Reform-Based Classroom. Lakehead University. Retrieved from <https://knowledgecommons.lakeheadu.ca/bitstream/handle/2453/4235/StewartA2018m-1b.pdf?sequence=1&isAllowed=y>.
Gersten, Chard, Jayanthi, Baker, Morphy, Flojo. (2009). A Meta-analysis of Mathematics Instructional Interventions for Students with Learning Disabilities: Technical Report. Instructional Research Institute. Retrieved from <http://3evoie.org/telechargementpublic/usa/gersten2009a.pdf?fbclid=IwAR0c-XjNJoSNy2dDvfWEwOqBl5EqtuFpU5GkW6s4QM7-jpuY90-I85Q5dyI>.
M, Duffy. (2017). Can Frequent Use Of Number Talks Increase The
Comprehension, Understanding, And Fluency Of Fractions, Decimals, And Percentages In Alternative High School Students? Hamline University. Retrieved from <https://digitalcommons.hamline.edu/cgi/viewcontent.cgi?article=5349&context=hse_all>.
May, P. L. (2020). Number Talks Benefit Fifth Graders’ Numeracy. International Journal of Instruction, 13(4), 361–374. https://doi-org.ezproxy.lakeheadu.ca/10.29333/iji.2020.13423a
Carbonneau, K. J., Marley, S. C., & Selig, J. P. (2013). A meta-analysis of the efficacy of teaching mathematics with concrete manipulatives. Journal of Educational Psychology, 105(2), 380-400. doi:http://dx.doi.org/10.1037/a0031084
J, Hattie. (2022). Meta-X. Visible Learning. Retrieved from <https://www.visiblelearningmetax.com/influences>.
Ran. (2022). A meta‐analysis on the effects of technology’s functions and roles on students’ mathematics achievement in K‐12 classrooms. Journal of Computer Assisted Learning., 38(1), 258–284.
Kacmaz. (2022). Examining pedagogical approaches and types of mathematics knowledge in educational games: A meta-analysis and critical review. Educational Research Review, 35, N.PAG.
SAYGILI, H., & ÇETİN, H. (2021). The Effects of Learning Management Systems (LMS) on Mathematics Achievement: A Meta-Analysis Study. Necatibey Faculty of Education Electronic Journal of Science & Mathematics Education, 15(2), 341–362. https://doi-org.ezproxy.lakeheadu.ca/10.17522/balikesirnef.1026534
Steenbergen-Hu, S., Makel, M. C., & Olszewski-Kubilius, P. (2016). What One Hundred Years of Research Says About the Effects of Ability Grouping and Acceleration on K–12 Students’ Academic Achievement: Findings of Two Second-Order Meta-Analyses. Review of Educational Research, 86(4), 849–899. https://doi.org/10.3102/0034654316675417
J, Hattie. (2022). Peer Tutoring. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/peer_tutoring>.
J, Hattie. (2022). RTI. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/response_to_intervention>
J, Hattie, (2022). Teaching to Learning Styles. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/matching_teaching_to_style_of_learning>.
Deunk. (2018). Effective differentiation Practices:A systematic review and meta-analysis of studies on the cognitive effects of differentiation practices in primary education. Educational Research Review, 24, 31–54.
E, Emanet, Et al.(2021). The Effects of Student-Centered Teaching Methods Used in Mathematics Courses on Mathematics Achievement, Attitude, and Anxiety: A Meta-Analysis Study. Participatory Educational Research. Vol 8(2), PP. 240-259.
J, Hattie. (2022). Student Centered Teaching. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/student-centered_teaching>.
J, Hattie. (2022). Problem Based Learning. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/problem-based_learning>.
J, Hattie. (2022). Collaborative Teaching. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/collaborative_learning>.
J, Hattie. (2022). Student Choice. Visible Learning Metax. Retrieved from <https://www.visiblelearningmetax.com/influences/view/school_choice>.
Disabilities: A Selective Meta-Analysis of the Literature. Learning Disability Quarterly, 44(4), 248–260. https://doi-org.ezproxy.lakeheadu.ca/10.1177/0731948721994843
J, Myers. (2022). A Meta-Analysis of Mathematics Word-Problem Solving Interventions for Elementary Students Who Evidence Mathematics Difficulties. The Review of Educational Research. Retrieved from <https://journals-sagepub-com.ezproxy.lakeheadu.ca/doi/full/10.3102/00346543211070049>.
Cason, M., Young, J., & Kuehnert, E. (2019). A meta-analysis of the effects of numerical competency development on achievement: Recommendations for mathematics educators. Investigations in Mathematics Learning, 11(2), 134–147. https://doi.org/10.1080/19477503.2018.1425591
Ennis, R. P., & Losinski, M. (2019). Interventions to Improve Fraction Skills for Students With Disabilities: A Meta-Analysis. Exceptional Children, 85(3), 367–386. https://doi-org.ezproxy.lakeheadu.ca/10.1177/0014402918817504
Carbonneau, K. J., Marley, S. C., & Selig, J. P. (2013). A meta-analysis of the efficacy of teaching mathematics with concrete manipulatives. Journal of Educational Psychology, 105(2), 380-400. doi:http://dx.doi.org/10.1037/a0031084
Fan. (2017). Homework and students’ achievement in math and science: A 30-year meta-analysis, 1986–2015. Educational Research Review, 20, 35–54.
Methe, S., Kilgus, S., Neiman, C., & Chris Riley-Tillman, T. (2012). Meta-Analysis of Interventions for Basic Mathematics Computation in Single-case Research. Journal of Behavioral Education, 21(3), 230–253. https://doi-org.ezproxy.lakeheadu.ca/10.1007/s10864-012-9161-1
Myers. (2021). Mathematics Interventions for Adolescents with Mathematics Difficulties: A Meta‐Analysis. Learning Disabilities Research & Practice., 36(2), 145–166.
R, Getsen, Et al. (2009). A Meta-analysis of Mathematics Instructional Interventions for Students with Learning Disabilities:
J, Myers. (2022). A Meta-Analysis of Mathematics Word-Problem Solving Interventions for Elementary Students Who Evidence Mathematics Difficulties. The Review of Educational Research. Retrieved from <https://journals-sagepub-com.ezproxy.lakeheadu.ca/doi/full/10.3102/00346543211070049>.
Ellington, A.J. (2003). A Meta-Analysis of the Effects of Calculators on Students' Achievement and Attitude Levels in Precollege Mathematics Classes. Journal for Research in Mathematics Education, 34, 433.