SUCCESS, A Program Providing Coaching and Financial Support to Community College Students

No-Spin’s Study Overview

Large, high-quality RCT of SUCCESS - a community college program that provides students with coaching, encouragement to enroll full-time, and financial incentives - finds no impacts on the study’s two primary outcomes: (i) total credits earned over one year (measured for the full sample), and (ii) credential completion over three years (measured for 72% of the sample - the early cohorts).

Program:

• SUCCESS is a community college program with four components: (1) frequent coaching for students; (2) encouragement to enroll full-time; (3) a $50 per month financial incentive; and (4) use of a data system to track student progress and support program management.
• SUCCESS was designed to be less expensive than other comprehensive student success programs, such as ASAP, by leveraging existing financial aid and offering smaller financial incentives and fewer support services. The program costs an average of $2,730 per student.

Study Design:

• The study sample comprised 4,189 students across 11 colleges. Individual students were randomly assigned to SUCCESS (treatment) or usual services (control). The sample was 28% Black, 31% Hispanic/Latino, and 46% “nontraditional” students.¹
• Based on our careful review, this was a high-quality RCT (e.g. baseline balance, no sample attrition, preregistered outcomes and analyses).

Findings on the Primary Outcomes:

• One year after random assignment, the study found no impact on cumulative credits earned. The treatment group earned an average of 16.3 credits versus an average of 16.2 credits for the control group – a difference that wasn’t statistically significant.
• Three years after random assignment, the study found no impact on completion of a college credential. Completion rates were 33.8% treatment versus 33.4% control – a difference that wasn’t statistically significant. (This outcome was measured for 72% of the sample - the early cohorts with three years of outcome data.)

Comment:

• Much of the study took place during the COVID-19 pandemic, requiring colleges to adapt the SUCCESS model – for example, by providing coaching virtually rather than in-person and, in many cases, dropping requirements for full-time enrollment. The study findings apply to the program as implemented under these conditions.
• We note, however, that another comprehensive student success program – ASAP, as delivered at Westchester Community College under the name "Viking ROADS" – was found to produce large impacts on college completion despite also being implemented and evaluated during the pandemic. One possible reason for Viking ROADS' greater impact is that it continued to enforce the full-time enrollment requirement, which may be critical to generating impacts.
• The SUCCESS RCT is ongoing and will report on longer-term outcomes in the future.
• Disclosure: Arnold Ventures, which funds No-Spin Evidence Review, helped fund the SUCCESS RCT.

¹ _{Nontraditional students are defined as those who were 25 or older, worked 35 or more hours per week, had children, or received a GED (instead of a high school diploma).}