Evidence Based Education

###### Eureka Math

Company Description:

Eureka Math is the most commonly used math curriculum in the United States. The authors market the product as “math minus memorization”, arguing “Thoughtfully constructed and designed like a story, Eureka Math is meticulously coherent, with an intense focus on key concepts that layer over time, creating enduring knowledge. Students gain a complete body of math knowledge, not just a discrete set of skills. They use the same models and problem-solving methods from grade to grade, so math concepts stay with them, year after year.” Which admittedly, I believe is marking that translates to the program emphasizes conceptual and application instruction but de-emphasize fluency instruction. The program offers both digital and print materials with the goal of making transitions to and from online learning seamless.

Curriculum:

Qualitative Factors that I Like:

-Sufficient sufficient fluency work

-The grade 6 curriculum included statistics

-A healthy blend of word problems and fluency instruction

-The program includes direct instruction components

-Most of the program components are free

Qualitative Factors that I Dislike:

-Word problems are two steps by grade 3. This seems inappropriately complex.

-Unit conversions are started in grade 4, this seems early

-Multi digit division in grade 4, this seems early

-Fractions operations in grade 4, this seems early

-Grade 5 curriculum introduces cartesian planes with word problems, before it introduces plotting.

-Algebra does not enter the curriculum before grade 6

-Not enough fractions instructions in the later grades

-Exponents are not introduced until grade 8

-Integers are not introduced until grade 8

Overall Impressions:

I was worried this program would have insufficient fluency instruction. However, I found the program to have a normal amount of fluency practice. That being said, much of the fluency work questions were phrased to help develop the students' math meta-cognition strategies, as well as their conceptual knowledge. However, personally I felt this sometimes made questions confusing and needlessly complex. Fluency practice is supposed to help students build computational and procedural automaticity. However, I worry that because the fluency questions were convoluted that it raised the potential to diminish the returns of the fluency work. For example, pre-algebra questions were phrased like: write three expressions that can = 12+3. The goal of this type of instruction is to help students to realize that the = sign actually means equivalent and not solve. However, I would argue that the more traditional method still achieves the same goal and does so in a less open-ended way.

Overall, I found the curriculum was too limited and that the scaffolding was for some strands too slow, while for others too fast. Personally, I would have liked to see a more spiraled curriculum, in which the same concepts were reviewed in every grade, but in increasing difficulties.

Research Base:

To the best of my knowledge there are no peer reviewed meta-analyses or studies on Eureka Math. The company does have 36 data reports listed on their website, which amount to essentially non peer reviewed case studies. I conducted a statistical analysis of these case studies to try to assess the efficacy of the program. However, I want to make several caveats to this analysis. Firstly, all of this research is compiled and published by the company, without peer review. Which means there is likely to be sponsorship bias, as there is nothing stopping the company from withholding all negative data and only posting their positive data. Secondly, not only is their data not peer reviewed, but neither is this analysis. Thirdly, there were many anomalies in this research, which were not necessarily malicious, but still limited the overall validity of the data for the purposes of establishing efficacy.

In total there were 36 case studies. Of these 36 case studies, 17 reported enough data, for me to calculate a statistical difference, with a mean total difference of 18.91%; however, 3 of these studies posted extreme outlier data, with a mean difference of more than 40%.If we correct for these outliers, we get a mean difference of 9.69%, which is not a particularly meaningful difference.

Of these 36 studies, 5 of these studies provided enough detail to calculate a Cohen’s d effect size. Admittedly, none of these studies had a true control group. But rather, they either provided comparison data based on the schools data from the year previous to the intervention or to state standards. Standard deviations (SD) were calculated based on the data between grades/schools and comparison groups. This is not a true SD and almost certainly inflated results. Of the studies, in which an effect size was calculated, several of them had longitudinal results. Effect sizes were always calculated based on 1 year of differences and not longitudinal results. Of the 31 studies that were excluded from this analysis, most were excluded for either not included data or including an inappropriate comparison group. A small number of studies were excluded because they did not produce exact figures, but rather used an unlabelled graph.

Studies Included in Effect Size Calculations:

This study was conducted in Louisiana and included 22000 grade 3-8 students. This study used state scores as a comparison group and had a mean ES of .54.

2: https://gm.greatminds.org/math/blog/eureka/vermilion-parish-la-continues-to-make-strong-progress

This study was also conducted in Louisiana and included 9500 grade 5-8 students. This study used test scores from the previous year as a comparison. This study had a mean ES of .93

3. https://gm.greatminds.org/math/blog/eureka/path-to-student-mastery-begins-with-teacher-mindset-shift

This study was conducted in New Mexico and included 18000 grade 6-9 students. This study used test scores from the previous year as a comparison.This study had a mean ES of 1.26

4: https://gm.greatminds.org/math/blog/eureka/going-all-in-pays-off-for-early-eureka-math-adopter

This study was conducted in Washington and included 9000 grade 3-5 students. This study used state scores as a comparison. This study found a mean ES of .20.

5.https://gm.greatminds.org/math/blog/eureka/grassroots-effort-spurs-excellence-outside-wichita This study was conducted in Wichita on 7678 grades kindergarten to grade 5 students. This study used test scores from the previous year as a comparison. This study had a mean ES of 2.49 and was likely an outlier due to an extremely small SD. The mean difference was only 11%.

In general, I would call these results impressive. However, the quality of evidence is extremely low. Overall, I would view this program as quasi-constructivist. The program does include quite a bit of direct instruction and fluency work, which I would argue make it less constructivist than other programs, like Marian Small’s MathUp. However, the program does have some constructivist elements in how it phrases math questions. Ultimately, I think it is very challenging to evaluate the efficacy of this program in a meaningful, objective way, as (to the best of my knowledge) there exists no high quality research on the subject. Moreover, I am not aware of any meta-analyses that adequately encompass this specific style of instruction.

Final Overall Grade: B

The program includes some research based elements and does have a high mean effect size in research. However, the research is of too low quality for results to be meaningful. Direct instruction, individualization, proper scaffolding, grade appropriate word problems, conceptual instruction, procedural instruction, fluency instruction and the inclusion of all math strands.

Final Qualitative Grade: 8/8

The program includes all essential elements including:

Written by Nathaniel Hansford

Last Edited 2022-06-28

References:

J, Taylor. (2022). Eureka Math Case Studies. Great Minds. Retrieved from <https://greatminds.org/case-studies?products=Eureka+Math&district_size=show-all&geography=show-all&grade_level=show-all&state=show-all&search=>

J, Taylor. (2015). Iberia and Vermilion, LA Schools Post Impressive Math Gains, Cite Eureka-Based Collaboration. Great Minds. Retrieved from <https://gm.greatminds.org/math/blog/eureka/iberia-and-vermilion-la-schools-post-impressive-math-gains-cite-eureka-based-collaboration>.

J, Taylor. (2017). Vermilion Parish (LA) Continues to Make Strong ProgressGreat Minds. Retrieved from <ttps://gm.greatminds.org/math/blog/eureka/vermilion-parish-la-continues-to-make-strong-progress>.

J, Taylor. (2018). Path to Student Mastery Begins with Teacher Mindset Shift. Great Minds. Retrieved from

<https://gm.greatminds.org/math/blog/eureka/path-to-student-mastery-begins-with-teacher-mindset-shift>.

J, Taylor. (2017). Vermilion Parish (LA) Continues to Make Strong Progress. Retrieved from

<https://gm.greatminds.org/math/blog/eureka/vermilion-parish-la-continues-to-make-strong-progress>

J, Taylor. (2015). Iberia and Vermilion, LA Schools Post Impressive Math Gains, Cite Eureka-Based Collaboration. Great Minds.

.