Randomized Controlled Trial (RCT) of the...
Chicago Heights Early Childhood Center (CHECC) Preschool Program
Reviewed
An RCT of the Chicago Heights (CHECC) preschool program, published in Journal of Political Economy (December 2024), portrays results of its grade-school follow-up as positive despite finding no impact on the preregistered primary outcome – state test scores in English and math.
Abstract
We investigate how skills developed when children are 3–5 years old drive schooling outcomes in middle childhood and adolescence. We find that skills map onto three distinct factors—cognitive skills, executive functions, and economic preferences. Importantly, each of the three factors predict later schooling outcomes. While early executive function skills and cognitive scores are linked to future behavioral patterns and other key student outcomes, economic preferences have an independent effect: children who are impatient in early childhood have more disciplinary referrals. Finally, random assignment to preschool impacts grades and disciplinary referrals through changes to cognitive skills and executive functions.
Excerpt from Study Introduction
Our fourth finding is that children randomly assigned to CHECC’s free, full-day preschool program have significantly higher grades and significantly lower disciplinary referrals than the control group. Both executive functions and cognitive skills mediate this relationship. Preschool increases cognitive skills and executive functions, which in turn explain 89% of the increase in grades. The mediation for disciplinary referrals is more complex. While increases in executive functions decrease disciplinary referrals (explaining 47% of the treatment effect), increases in cognitive skills actually increase disciplinary referrals (explaining 8% of the relationship).
We investigate how skills developed when children are 3–5 years old drive schooling outcomes in middle childhood and adolescence. We find that skills map onto three distinct factors—cognitive skills, executive functions, and economic preferences. Importantly, each of the three factors predict later schooling outcomes. While early executive function skills and cognitive scores are linked to future behavioral patterns and other key student outcomes, economic preferences have an independent effect: children who are impatient in early childhood have more disciplinary referrals. Finally, random assignment to preschool did not have a discernible impact on the primary study outcome in grade school—state test scores in English language arts and math. In exploratory analyses, we found that preschool impacts grades and disciplinary referrals through changes to cognitive skills and executive functions. Study limitations include high sample attrition for grade-school outcomes.
No-Spin’s Study Overview
An RCT of the Chicago Heights Early Childhood Center (CHECC) preschool program found no impact on children’s grade school test scores (the primary outcome), but the findings are only suggestive due to high sample loss.
Program and Study Design:
- The CHECC study randomized 1,740 low-income children aged 3-4 to (i) 9 months of full-day preschool vs (ii) control group. The study design was complex – with other treatment arms, sub-RCTs, and a subsequent randomization in grade school.
- But the study's preregistered primary outcome for the grade-school follow-up of CHECC preschool was straightforward: state test scores (an average of Math and English Language Arts) in 3rd grade for the full preschool group vs the full control group.
Findings:
- The study reports on an outcome slightly different than that preregistered as primary (namely, test scores in grades 3 & 4, not just grade 3). The study found no impact on this outcome.
- The study also found no impact on this outcome for a separate parent training program (Parent Academy) tested in another arm of the RCT.
- The study found positive impacts for the preschool program on two exploratory outcomes: student grades and disciplinary referrals in grades 3-8. These findings may offer good hypotheses to test in future studies, but are only preliminary – not reliable – under established standards (FDA, IES), as they could be false-positives resulting from the study's analysis of dozens of outcomes, sub-RCTs, and subgroups.
Comment:
- We selected this study for a No-Spin report to illustrate the problem in its portrayal of the results. But we'd also note that the study had sample loss (i.e. attrition) of approximately 50%, with differentially higher attrition in the control group. Thus, we believe its findings, including the null primary results, are only suggestive.
- Disclosure: Arnold Ventures, which funds the No-Spin Evidence Review, helped fund the CHECC RCT.
Click or tap a highlight to see No-Spin’s comment