A Randomized Controlled Trial (RCT) of...

Accelerate, Complete, Engage (ACE), Providing Supports for College Students to Complete a 4-Year Degree

Reviewed

Postsecondary Education
Accurately reported
Credible positive finding

Metis Associates and the City University of New York (March 2024) published an RCT of Accelerate, Complete, Engage (ACE) at John Jay College. ACE, adapted from the Accelerated Study in Associate Programs (ASAP) model, provides comprehensive supports for mostly low-income, four-year college students. This high-quality RCT found a remarkable 12 percentage point gain in bachelor's degree completion after five years.

Relevant Excerpts from the Executive Summary:

In fall 2015, the City University of New York (CUNY) piloted Accelerate, Complete, Engage (ACE), a comprehensive student success program for bachelor’s degree-seeking students at John Jay College of Criminal Justice. The ACE program model was based on and closely mirrored the Accelerated Study in Associate Programs (ASAP), a program that has been demonstrated to be highly successful for associate degree-seeking students. This study, a randomized controlled trial (RCT), was designed to test whether the ACE program impacted four-year and five-year graduation rates for students who entered as freshmen and to explore whether effects vary by subgroups defined by sex, race/ethnicity, and socioeconomic status.

*       *       *

Findings:

  • ACE students were estimated to graduate at a rate of 68.8%, 11.7 percentage points higher than the control group rate of 57.1%. This estimated impact was associated with a small effect size (Cox index = 0.307) and considered substantively important.
  • The estimated five-year graduation rates were approximately 10 percentage points higher for both groups than the estimated four-year graduation rates (58.8% for the program group vs. 46.4% for the control group), with an impact similar in size (11.7 percentage points vs. 12.4 percentage points).
  • The observed difference in graduation rates between ACE and control students was almost twice as large for male students (15.7 percentage points) than female students (8.9 percentage points), although female students continued to graduate at higher rates (72.9%) than male students (58.1%).
  • Black, Hispanic or Latino/a, and White students displayed similar graduation rates (67.9%, 66.9%, and 64.9%, respectively). However, the observed differences between ACE students and controls were almost three times as high for Black (17.9 percentage points) and Hispanic or Latino/a (16.6 percentage points) students than White students (5.8 percentage points). Asian or Pacific Islander students had the highest five-year graduation rates at 80%, with a difference of 8 percentage points between ACE and control students.

Full Study Report

We have no suggested revisions to the study’s published executive summary.

No-Spin’s Study Overview

High-quality RCT of Accelerate, Complete, Engage (ACE) at John Jay College – a program that provides comprehensive supports for mostly low-income, four-year college students – finds a remarkable 12 percentage point gain in bachelor’s degree completion after five years.

Program:

  • ACE is modeled on ASAP – a community college program found in prior RCTs to substantially increase graduation rates. ACE provides the same academic, personal and financial supports adapted to a four-year institution – John Jay College. Both programs require students to enroll full-time. The direct cost of delivering ACE as part of the study was approximately $12,400 per student.1

Study Design:

  • The study sample comprised 570 incoming freshmen, who were randomly assigned to ACE (treatment group) versus services-as-usual (control group). The sample was 70% female and 58% Black or Hispanic. Average parent income was $48,000 per year. For 61% of the sample, neither parent had gone to college.
  • Based on our review, this was a high-quality RCT (e.g., baseline balance, no sample attrition, preregistered analyses).

Findings:

  • Five years after program entry, the study found that ACE increased the rate of graduation with a bachelor’s degree by 12 percentage points. Specifically, 69% of the treatment group graduated vs. 57% of the control group – a difference that was  statistically significant (p<0.01).

Comment:

  • The two sister programs – ACE (for four-year college students) and ASAP (for two-year community college students) – have been found to produce large impacts on degree completion across four high-quality RCTs.
  • Arnold Ventures, which funds No-Spin Evidence Review, funded the ACE RCT.

1 The direct costs of delivering ACE program services at John Jay College were estimated to be $12,374 per student (2023 dollars). Taking into account both the direct costs and the indirect costs resulting from the fact that treatment group students earned more credits than control group students, the per-student cost estimate was $15,694.

Click or tap a highlight to see No-Spin’s comment

Related Reviews