Randomized Controlled Trial (RCT) of...
City Year’s Whole School Whole Child Tier 2 Services for At-risk Middle School Students
Reviewed
Journal of Research on Educational Effectiveness (September 2024) published an RCT of City Year Whole School Whole Child (WSWC) Tier 2 services for at-risk middle school students, consisting of tailored assistance (e.g., tutoring, mentoring) delivered by Student Success Coaches. This high-quality RCT – evaluating the added benefit of Tier 2 services for at-risk students over City Year’s schoolwide assistance – found no discernible impacts on student academic or behavioral outcomes over 1 - 1.5 years.
The paper reports results from an evaluation of City Year's Whole School Whole Child (WSWC) model's Tier 2 services in 22 middle schools in five urban school districts before the COVID-19 pandemic. Student Success Coaches provide universal holistic services and Tier 2 services to students based on assignment to one or more “focus lists” of students recommended based on one or more early indicators that are associated with whether students will stay in school and graduate from high school. The effects of Tier 2 services on student math and ELA achievement, attendance, social-emotional skills, and suspensions were evaluated using a two-year, individual randomized controlled trial (RCT). The sample included 1,755 Grade 6 students randomly assigned to receive Tier 2 services or to the control group. There were no significant effects of Tier 2 services on student outcomes. Limitations of the study, including disruptions due to COVID-19 and the challenges of implementing an RCT as a method for studying a complex intervention, are discussed.
We have no suggested revisions to the published summary.
No-Spin’s Study Overview
High-quality RCT of City Year's WSWC Tier 2 services for at-risk middle school students, evaluating the added benefit of these services over City Year’s schoolwide assistance, finds no discernible impacts on student academic or behavioral outcomes over 1 - 1.5 years.
Program:
- Under City Year's Whole School Whole Child (WSWC) model, young adult AmeriCorps members serve as Student Success Coaches in schools, providing (i) schoolwide (“Tier 1”) services - such as after-school programs, family engagement events, and in-classroom support - to all students; and (ii) targeted (“Tier 2”) services - such as one-on-one and small group tutoring and mentoring - to students at risk of not graduating based on early warning indicators.
Study Design:
- The study sample comprised 1,755 at-risk 6th graders in 22 urban middle schools across the U.S. that were receiving schoolwide Tier 1 services. The students were randomly assigned to treatment (Tier 2 services, in addition to Tier 1) versus control (Tier 1 only). 58% of sample members were Latino, 36% were Black, and 94% were low income.
- The study evaluated the Tier 2 services as delivered over 1.5 school years (SY 2018-19 and fall 2019), after which the study ended due to the COVID pandemic.
- Based on careful review, this was a well-conducted RCT (e.g., baseline balance, low sample attrition for most outcomes, high-quality implementation of Tier 2 services in the treatment group).
Findings:
- The study found no discernible impacts on any student outcome – including English language arts (ELA) and math scores on the state test measured at the end of one school year; or ELA and math grades, attendance, and disciplinary suspensions measured over 1.5 school years.
Click or tap a highlight to see No-Spin’s comment