Recently, Dr. Rachel Schechter and I have been doing research together to answer the question, can 95% of students learning how to read and if so, how do we achieve this goal. As part of this research, I was specifically trying to find the impact of class size. I realized that I could use the NAEP database, which collects test scores for every state in the US. This enormous database allows us to create some natural experiments, and answer some otherwise, very difficult questions. Using this data has some pretty unique advantages and disadvantages. On the plus side, it allows the opportunity to retrospectively create studies, with enormous sample sizes. Even better, unlike within a meta-analysis, the measurement is always the same, which makes the results more universally comparable. On the negative side, there is very little information available about what education looks like for individual students.
I decided to start by looking at the impact of student to teacher ratios. I would have preferred to look at class sizes, but the NAEP database does not have that information. For each state, I recorded the 2023 NAEP raw scores for grade 4 literacy, the percentage of students that met the basic proficiency benchmark for the test, and the student to teacher ratio, within a spreadsheet. Then I conducted a Pearson correlation test, which produces an effect size for the degree of correlation, between the two factors. The results can be seen below.
The above results show a very weak, albeit technically statistically significant relationship between the student to teacher ratio and reading achievement in grade 4. One criticism I have seen of this type of analysis before, is that it only looks at the overall trend and does not break down the results, for greater deviations in class size from the norm. I decided to redo-this analysis, I broke down student to teacher ratios into 8 categories. I used the national average of 14.85, as a control sample and calculated a Cohen’s d effect size for each category, by comparing the difference in achievement for that group in comparison to the national average. The results can be seen below.
The highest performing ratio was 12:1 students per teacher. In fact, this was the only group of states that outperformed the national average by a non-negligible amount. However, the 11:1 group of states did far worse and the group of states with the highest ratio actually tied the national average. Overall, it does not appear to me that class size has a strong correlation on student achievement. Of course, logically this seems very unintuitive. All other factors equal, we would assume that class size has a tremendous effect on student outcomes. A lower number of pupils would mean more time to spend providing one on one instruction. It also means the teacher has less marking and classroom management to do, which would presumably free up time for planning and intervention.
So, what’s going on here? I wondered if income had an impact on this and repeated the correlation analysis, but this time compared the state GDP to achievement outcomes. To my immense surprise, I found a negative (albeit statistically insignificant) correlation between state income and student achievement. This means that on average wealthier states do worse on the NAEP than poorer states. Again, this seems highly counterintuitive. I would imagine that if we polled people on the street what factors most influence education outcomes they would guess, class size and income.
So why do these factors seem to be producing such low impacts? Is it that they don’t matter or is it that other factors more? My guess is that it’s the latter. I would presume that the more explicit the instruction and the more specific the instruction is to student learning needs the higher the achievement outcomes will be. The real advantage of small class sizes would be increased time for explicit instruction, specified to student needs. However, if the reduced class size is not used to increase the explicitness or specificity of instruction, I would imagine that the impact is much smaller. I would also hypothesize that if all other factors are equal that class size does make a significant difference; however, I think this is muddied in research, by the fact that instruction quality has a far greater impact on student achievement.
Written by Nathaniel Hansford and Contributed to and sponsored by Dr. Rachel Schechter of LXD Research.
Last Edited 2023-04-29
About LXD Research LXD Research is an independent evaluation, research, and consulting division within Charles River Media Group, LLC focusing on educational technology. We design rigorous research studies, multifaceted data analytic reporting, and dynamic content to disseminate insights. Visit www.LXDResearch.com
NAEP. (2023). The Nation’s Report Card. https://www.nationsreportcard.gov/