Evidence Reports

Third Sector posted (August 2024) RCT findings for a Massachusetts Pay for Success Project that provided Roca – a violence intervention and behavioral health program – to justice-involved young men to prevent reoffending. This high-quality RCT found no discernible effects on reincarceration or employment over 5 years. The study was able to rule out the sizable positive effects anticipated as the project’s start, but not more modest effects.

Mobility posted (October 2024) long-term RCT results for Project QUEST's occupational training of low-income adults for well-paying healthcare jobs. This high-quality RCT found a 15-20% gain in annual earnings at the 14 year follow-up - an extremely promising (albeit not quite definitive) result.

The Annenberg Institute (October 2024) posted an RCT of Choose to Change, a group therapy and mentoring program for at-risk teens. Despite the study abstract’s claimed effects on youth crime, the study found no discernible impact on its preregistered primary measure of crime – number of youth arrests – over 3 years.

An RCT of a cash transfer program for individuals experiencing homelessness, published in the August 2023 Proceedings of the National Academies of Sciences (PNAS), portrays its results as positive despite finding no discernible impact on any of the study’s pre-registered primary or secondary outcomes.

NBER (April 2024) posted an RCT of Khoaching with Khan Academy – a program integrating computer-assisted learning into classroom math teaching. Despite the positive portrayal of results in the study’s abstract, it found no discernible impact on the prespecified primary outcome – grade 3-8 math scores on the state test after one year.

Journal of Research on Educational Effectiveness (Sept 2024) published an RCT of City Year's WSWC Tier 2 services (e.g., tutoring, mentoring) for at-risk middle school students. This high-quality RCT, evaluating the added benefit of Tier 2 services over City Year’s schoolwide assistance, found no discernible impacts on student outcomes over 1-1.5 years.

NBER (May 2024) posted an RCT of Saga Technology, a program that provides 9th graders with high-dosage math tutoring and engagement with educational technology. This large, high-quality RCT found sizable gains (~86% of a grade-level) in math achievement on end-of-year district tests.

An RCT of the Chicago Heights (CHECC) preschool program, published in Journal of Political Economy (December 2024), portrays results of its grade-school follow-up as positive despite finding no impact on the preregistered primary outcome – state test scores in English and math.

Development & Psychopathology (December 2022) published an RCT of a program that provides home visiting and doula visits to low-income women starting in pregnancy. Despite the positive portrayal of results in the study’s abstract, the study found no discernible impact on child outcomes and little impact on parent practices, at age 30 months.