Evidence Based Education

###### Secondary Math Instruction

Long-time readers will know that it is my opinion that the efficacy of a pedagogical strategy or factor cannot be determined without meta-analysis. Generally speaking, meta-analysis effect sizes are supposed to be viewed with the following interpretation.

However, research is always context-specific. A single meta-analysis can find an effect size of .45 on a topic, but that does not necessarily mean that the importance of that effect is moderate. We need to be able to identify how those factors interrelate to other factors within the same context. For example, my recent non-peer-reviewed meta-analysis on phonics found a mean ES of structured Literacy of .45, which is a moderate effect. However, it is significantly higher than the effect found for Balanced Literacy and Whole Language within the meta-analysis literature. As these are the most popular approaches to language instruction, we can say that Structured Literacy is the most evidence-based approach of the three, despite having only a moderate effect size.

For this reason, it can be useful to conduct a secondary meta-analysis, where the results of multiple meta-analyses are put into a single meta-analysis, to better allow comparisons between different pedagogies, within the same context. Recently, I conducted a secondary meta-analysis on the topic of math instruction, which looked at 41 meta-analyses and studies, to attempt at ranking math pedagogies and factors based on their effect size. Although truthfully, I think the greatest weakness of this analysis lies in the fact that the impact of pedagogical factors tends to be age-dependent, meaning that the pedagogies that work for one age do not work for another. For example, research tends to show that situational problems produce low results in younger students than older ones. To this end, I have endeavored to break down the same meta-analysis data, by age. In this article, I have compiled all the meta-analysis research I have, of secondary-aged (grades 9-12) students, into a single secondary meta-analysis.

Results:

Discussion:

I think one obvious caveat we need to make about this information is that we obviously have a deficit of research on specifically how to teach secondary students math. As the majority of math instruction meta-analyses, do not specifically look at this issue. Specifically, I think we have a deficit of information on math fluency, and conceptual instruction for secondary students. I also think we need more information regarding explicit vs implicit math instruction in this age range, and on lesson formats.

Those caveats aside, we do appear to have strong evidence for game-based learning, student centered teaching, mathematical modeling, and cooperative motivation for this age range. I must admit, these results surprised me. When I expected game-based learning to be most effective for primary students, specifically regarding their math fluency results. However, we see much lower results for primary students and game based learning. Moreover, we do not see the highest primary results for number fact games. Inversely, I would have thought that game based learning would be ineffective for this age range, as I would have expected secondary math to be complicated for games to be useful instructional tools. A colleague of mine hypothesized that the difference might be in how the students perceive the games. While primary students might focus on winning the game, for the sake of winning the game, secondary students might see it as a tool to help them learn.

One key distinction between secondary students and elementary students appears to be that secondary students appear to benefit from complex situational problems, whereas elementary students do not. Perhaps this is because secondary students on average have more sophisticated levels of foundational knowledge that in turn makes application easier.

Factor Glossary:

Word Problem Instruction: This effect size was from the 2022 Myers meta-analysis. For more information, you can read this article: https://www.teachingbyscience.com/word-problems

Number Fact Instruction: Instruction on basic arithmetic, IE adding, subtracting, multiplying, dividing. This effect size is sourced from the 2012 Scott Methe Meta-analysis. For more information check out: https://www.teachingbyscience.com/math-fact-fluency

Experiential Games: “Learning and teaching in games are based on learning by doing and solving real-life problems through experiencing and interacting with the environment. Learners gain understanding by engaging in simulated actions related to real-life experiences and learn by interacting with the objects in the game. The fundamental basis for experiential learning is the active role of the learner through interaction with the environment.” (Kacmaz) This effect size was sourced from the 2022 Kacmaz, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/game-based-learning-in-math

DI Games: “Learning is linked with stimulus-response conditioning, rapid-pace drills, or structured lesson plans that generate student engagement through pacing and immediate feedback. Learning and instruction that entails rote memorization of facts and does not necessarily facilitate creative thought. The presentation of the game follows the question, answer, and feedback. Repetitive practice is offered.” This effect size was sourced from the 2022 Kacmaz, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/game-based-learning-in-math

Cooperative Motivation: This effect size is based on using cooperative motivation strategies vs competitive ones, it is sourced from the 1995 Zhinning meta-analysis. For more information check out: https://www.teachingbyscience.com/cooperative-vs-competitive-education

Manipulatives: This effect size is sourced from the Carbonneau, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/manipulatives

Game-Based Learning: This effect size was sourced from the 2022 Kacmaz, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/game-based-learning-in-math

Homework: This effect size was sourced from the 2018 Fan, Et al meta-analysis. For more information check out: https://www.teachingbyscience.com/math-homework

Technology: This effect size is based on the generalizable impact of technology and is sourced from the 2022 Ran, Et al meta-analysis. For more information check out https://www.teachingbyscience.com/technology-and-math

Calculators: This effect size is based on the impact of giving students calculators and is sourced from the Aimee Ellington 2003 meta-analysis. For more information check out: https://www.teachingbyscience.com/calculator

Mathematical Modeling: “The process of encountering an indeterminate situation, problematizing it, and bringing inquiry, reasoning, and mathematical structures to bear to transform the situation. The modeling produces an outcome - a model - which is a description or a representation of the situation, drawn from the mathematical disciplines, in relation to the person's experience, which itself has changed through the modeling process.” This effect size is sourced from the 2015 Sokolowski meta-analysis.

Discussion:

I think we see a few interesting trends, within the intermediate grades. Firstly, the effects of math fact instruction and manipulatives appear to show clear diminishing returns. Indicating that perhaps this type of instruction should start to lessen, in this grade. On the other hand, we see strong evidence still for the use of word problems, and student-centered teaching (although I must admit, I have concerns with the contexts of the findings for this meta-analysis). Perhaps most unusual, is that we see a very strong benefit for not letting intermediate students use calculators. This is particularly interesting because we do not see the same benefit in any other set of grades.

I do think it makes sense that we see lower impacts for number fact instruction and manipulatives in this age range, as these types of instruction are for the basic instruction of conceptual and fluency understanding. However, we do not have particularly useful data on how best to teach grade-appropriate fluency and conceptual knowledge, for this age range.

Written by Nathaniel Hansford

Last Edited: 2022-05-20

References:

Sokolowski, A. (2015). The Effects of Mathematical Modeling on Students’ Achievement-Meta-Analysis of Research. IAFOR Journal of Education, 3(1), 93–114. https://doi-org.ezproxy.lakeheadu.ca/10.22492/ije.3.1.06

Kacmaz. (2022). Examining pedagogical approaches and types of mathematics knowledge in educational games: A meta-analysis and critical review. Educational Research Review, 35, N.PAG.

Deunk. (2018). Effective differentiation Practices:A systematic review and meta-analysis of studies on the cognitive effects of differentiation practices in primary education. Educational Research Review, 24, 31–54.

E, Emanet, Et al.(2021). The Effects of Student-Centered Teaching Methods Used in Mathematics Courses on Mathematics Achievement, Attitude, and Anxiety: A Meta-Analysis Study. Participatory Educational Research. Vol 8(2), PP. 240-259.

Carbonneau, K. J., Marley, S. C., & Selig, J. P. (2013). A meta-analysis of the efficacy of teaching mathematics with concrete manipulatives. Journal of Educational Psychology, 105(2), 380-400. doi:http://dx.doi.org/10.1037/a0031084

Qin, Z., Johnson, D. W., & Johnson, R. T. (1995). Cooperative Versus Competitive Efforts and Problem Solving. Review of Educational Research, 65(2), 129–143. https://doi.org/10.3102/00346543065002129

Fan. (2017). Homework and students’ achievement in math and science: A 30-year meta-analysis, 1986–2015. Educational Research Review, 20, 35–54.

Ran. (2022). A meta‐analysis on the effects of technology’s functions and roles on students’ mathematics achievement in K‐12 classrooms. Journal of Computer Assisted Learning., 38(1), 258–284.

Ellington, A.J. (2003). A Meta-Analysis of the Effects of Calculators on Students' Achievement and Attitude Levels in Precollege Mathematics Classes. Journal for Research in Mathematics Education, 34, 433.

Methe, S., Kilgus, S., Neiman, C., & Chris Riley-Tillman, T. (2012). Meta-Analysis of Interventions for Basic Mathematics Computation in Single-case Research. Journal of Behavioral Education, 21(3), 230–253. https://doi-org.ezproxy.lakeheadu.ca/10.1007/s10864-012-9161-1

Cason, M., Young, J., & Kuehnert, E. (2019). A meta-analysis of the effects of numerical competency development on achievement: Recommendations for mathematics educators. Investigations in Mathematics Learning, 11(2), 134–147. https://doi.org/10.1080/19477503.2018.1425591