Evidence Based Education
What is Current Best Practice for Junior Students?
Long time readers will know that it is my opinion that efficacy for a pedagogical strategy or factor cannot be determined without meta-analysis. Generally speaking meta-analysis effect sizes are supposed to be viewed with the following interpretation.
However, research is always context specific. A single meta-analysis can find an effect size of .45 on a topic, but that does not necessarily mean that the importance of that effect is moderate. We need to be able to identify how those factors interrelate to other factors within the same context. For example, my recent non-peer reviewed meta-analysis on phonics found a mean ES of structured Literacy of .45, which is a moderate effect. However, it is significantly higher than the effect found for Balanced Literacy and Whole Language within the meta-analysis literature. As these are the most popular approaches to language instruction, we can say that Structured Literacy is the most evidence based approach of the three, despite having only a moderate effect size.
For this reason, it can be useful to conduct a secondary meta-analysis, where the results of multiple meta-analyses are put into a single meta-analysis, to better allow comparisons between different pedagogies, within the same context. Recently, I conducted a secondary meta-analysis on the topic of math instruction, which looked at 41 meta-analyses and studies, to attempt at ranking math pedagogies and factors based on their effect size. Although truthfully, I think the greatest weakness of this analysis lies in the fact that the impact of pedagogical factors tends to be age dependent, meaning that the pedagogies that work for one age do not work for another. For example, research tends to show that situational problems produce low results in younger students than older ones. To this end, I have endeavored to break down the same meta-analysis data, by age. In this article, I have compiled all the meta-analysis research I have, of junior-aged (grades 3-5) students, into a single secondary meta-analysis.
Results:
Factor Glossary:
Number Fact Instruction: Instruction on basic arithmetic, IE adding, subtracting, multiplying, dividing. This effect size is sourced from the 2012 Scotte Methe Meta-analysis. For more information check out: https://www.teachingbyscience.com/math-fact-fluency
Experiential Games: “Learning and teaching in games are based on learning by doing and solving real-life problems through experiencing and interacting with the environment. Learners gain understanding by engaging in simulated actions related to real-life experiences and learn by interacting with the objects in the game. The fundamental basis for experiential learning is the active role of the learner through interaction with the environment.” (Kacmaz) This effect size was sourced from the 2022 Kacmaz, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/game-based-learning-in-math
Fluency Instruction: Math fluency is defined as speed and accuracy. Instruction in this regard attempts to increase students' math fluency, via direct instruction and practice. This effect size is sourced from the 2019 Cason, Et al meta analysis. For more information check out: https://www.teachingbyscience.com/math-fluency
DI Games: “Learning is linked with stimulus-response conditioning, rapid-pace drills, or structured lesson plans that generate student engagement through pacing and immediate feedback. Learning and instruction that entails rote memorization of facts and does not necessarily facilitate creative thought. The presentation of the game follows question, answer, and feedback. Repetitive practice is offered.” This effect size was sourced from the 2022 Kacmaz, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/game-based-learning-in-math
Cooperative Motivation: This effect size is based on using cooperative motivation strategies vs competitive ones, it is sourced from the 1995 Zhinning meta-analysis. For more information check out: https://www.teachingbyscience.com/cooperative-vs-competitive-education
Manipulatives: This effect size is sourced from the Carbonneau, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/manipulatives
Game Based Learning: This effect size was sourced from the 2022 Kacmaz, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/game-based-learning-in-math
Homework: This effect size was sourced from the 2018 Fan, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/math-homework
Constructivist Games: “Learners are actively engaged in their own learning such that knowledge is assumed to be constructed by learners rather than transmitted. Constructivism closely relates to experiential and discovery learning. However, it adds the construction of personal meaning by the learner as a final step.” (Kacmaz) This effect size was sourced from the 2022 Kacmaz, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/game-based-learning-in-math
Technology: This effect size is based on the generalizable impact of technology and is sourced from the 2022 Ran, Et al meta-analysis. For more information check out https://www.teachingbyscience.com/technology-and-math
Calculators: This effect size is based on the impact of giving students calculators and is sourced from the Aimee Ellington 2003 meta-analysis. For more information check out: https://www.teachingbyscience.com/calculator
Discovery Based Games: “Learning occurs as students discover concepts on their own through levels. Discovery learning builds on existing knowledge to discover new things, the learner applies inquiry-based reasoning, performs problem solving, makes the decision, and applies strategy. Students interact with games by exploring and manipulating objects or performing experiments”. This effect size was sourced from the 2022 Kacmaz, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/game-based-learning-in-math
Discussion:
I think this research has several limitations, the first and most important being, there really is not enough research in this area. While meta-analysis is important, I would argue that data can often be very age dependent; however, we clearly do not have much meta-analysis data on junior aged students. While this data does suggest a high importance for fluency instruction at this age, I would argue there are two limitations to this point. Firstly, the 2012 Scotte Methe meta-analysis (the source for this ES) was a single case-meta analysis, which limits the reliability of the results; moreover, very little of this meta-analysis data is on conceptual, or application based instruction, which makes it hard to compare and contrast the value of conceptual vs procedural instruction in this age group.
That being said, we do see the highest manipulatives results for this age range, possibly showing that the best time to use manipulatives is for the junior grades. Moreover, while we see very high outcomes for math facts instruction at this grade, we see lower results for teaching numeracy skills such as place value. To me this possibly suggests that while numeracy skills are most importantly developed in the primary grades, that we start to see some diminishing returns for teaching them in the junior grades.
Disclaimers aside, I think we see strong evidence for fluency instruction, math fact fluency instruction, and direct instruction/experiential games at this age. I also think we see moderate evidence for manipulatives and cooperative motivation at this age.
Written by Nathaniel Hansford
Last Edited 2022-04-29
References:
Kacmaz. (2022). Examining pedagogical approaches and types of mathematics knowledge in educational games: A meta-analysis and critical review. Educational Research Review, 35, N.PAG.
Deunk. (2018). Effective differentiation Practices:A systematic review and meta-analysis of studies on the cognitive effects of differentiation practices in primary education. Educational Research Review, 24, 31–54.
E, Emanet, Et al.(2021). The Effects of Student-Centered Teaching Methods Used in Mathematics Courses on Mathematics Achievement, Attitude, and Anxiety: A Meta-Analysis Study. Participatory Educational Research. Vol 8(2), PP. 240-259.
Carbonneau, K. J., Marley, S. C., & Selig, J. P. (2013). A meta-analysis of the efficacy of teaching mathematics with concrete manipulatives. Journal of Educational Psychology, 105(2), 380-400. doi:http://dx.doi.org/10.1037/a0031084
Qin, Z., Johnson, D. W., & Johnson, R. T. (1995). Cooperative Versus Competitive Efforts and Problem Solving. Review of Educational Research, 65(2), 129–143. https://doi.org/10.3102/00346543065002129
Fan. (2017). Homework and students’ achievement in math and science: A 30-year meta-analysis, 1986–2015. Educational Research Review, 20, 35–54.
Ran. (2022). A meta‐analysis on the effects of technology’s functions and roles on students’ mathematics achievement in K‐12 classrooms. Journal of Computer Assisted Learning., 38(1), 258–284.
Ellington, A.J. (2003). A Meta-Analysis of the Effects of Calculators on Students' Achievement and Attitude Levels in Precollege Mathematics Classes. Journal for Research in Mathematics Education, 34, 433.
Methe, S., Kilgus, S., Neiman, C., & Chris Riley-Tillman, T. (2012). Meta-Analysis of Interventions for Basic Mathematics Computation in Single-case Research. Journal of Behavioral Education, 21(3), 230–253. https://doi-org.ezproxy.lakeheadu.ca/10.1007/s10864-012-9161-1
Cason, M., Young, J., & Kuehnert, E. (2019). A meta-analysis of the effects of numerical competency development on achievement: Recommendations for mathematics educators. Investigations in Mathematics Learning, 11(2), 134–147. https://doi.org/10.1080/19477503.2018.1425591