Relevant Excerpt from the Study Abstract:

The 10-year findings shown in this report address a more ambitious question: Can short-term sec­tor programs generate earnings gains a full decade later? To answer this question, the analysis focuses on two prespecified confirmatory outcomes in Year 10: (1) total annual earnings and (2) the proportion of individuals earning $45,000 or more, both drawn from administrative data in the National Directory of New Hires. Because the study followed multiple cohorts, Year 10 cov­ers 2021 through 2023 and coincides with the COVID-19 pandemic, an unprecedented economic shock that impacted sectors in different ways and no doubt impacted the economic outcomes of the people in the study.

Findings

• In Year 10, the WorkAdvance program at St. Nicks Alliance increased average earnings by 32 percent and increased the proportion of people who earned $45,000 or more by 7 percent­age points.

• The other three WorkAdvance programs did not have an impact on either confirmatory out­come in Year 10.

Overall, the findings from this 10-year evaluation suggest that sector programs are a promising way to help individuals increase their earnings and advance in their careers. All four programs increased earnings at some point—the Towards Employment and Madison Strategies Group pro­grams in Year 2, the Per Scholas program in Years 2 to 5 and 7 to 8, and the St. Nicks Alliance program in Years 8 to 10—leaving people better off overall than they would have been without the programs.

Full Study Report

We have no suggested revisions to the study abstract.

No-Spin’s Study Overview

High-quality RCT of Per Scholas – an employment/training program for low-income adults, focused on Information Technology (IT) jobs – finds (i) sizable earnings gains over years 2-8 after study entry and cumulative earnings gains of 16% over 10 years;1 and (ii) minimal earnings gains (not statistically significant) in years 9-10, which could indicate fading effects or just reflect imprecision of long-term earnings estimates.

A related high-quality RCT of another job training program (St. Nicks Alliance) finds sizable earnings gains in years 8-10 that may be driven by shifts in the economy that would be hard to predict in advance, and thus may be difficult to replicate.   

Program:

  • Per Scholas provides participants with 15 weeks of occupational skills training in IT, career readiness services (e.g., resume and interviewing assistance), and job development and placement services. The program has strong relationships with local employers that hire workers with IT skills, and the employers help shape the program’s training curriculum and services. The program cost is approximately $8,000 per participant (2025 dollars).

Study Design:

  • The study randomly assigned 700 unemployed or low-wage workers in the Bronx to a treatment group that was offered Per Scholas services or to a control group that was not. 87% of sample members were male, 81% were Black or Hispanic, virtually all had a high school diploma or GED, and their average age was 31.
  • Based on careful review, this was a high-quality RCT (e.g., baseline balance, negligible sample attrition).  

Findings:

  • Per Scholas produced sizable, statistically significant earnings gains in years 2-8 after study entry, as shown in the graph below.2 The table beneath the graph shows that (i) the earnings gains were 14-15% in years 7-8 and (ii) the cumulative gains over 10 years were approximately 16% ($42,000), compared to the control group.
  • However, as shown, in years 9 and 10 the earnings impacts diminished in size and were no longer statistically significant.
  • The findings in years 9 and 10 could indicate fading effects. But they may also simply reflect chance variation in yearly impacts caused by the study’s reduced ability to precisely estimate earnings impacts as the follow-up period lengthens.4
  • In the same report, MDRC presented findings of a high-quality RCT of St. Nicks Alliance’s job training program, focused on environmental remediation, pest control, and other sectors. The study found no discernible earning gains in years 1-5 but sizable, statistically significant gains in years 8-10.
    • While intriguing, the earnings gains in years 8-10 may be driven by sectoral shifts in the economy, possibly related to the COVID-19 pandemic, that would have been very difficult to anticipate years in advance. For that reason, we believe the findings may not be actionable evidence to inform policy decisions about which job training programs to invest in.

Comment:

  • This is the second RCT of Per Scholas showing sizable earnings gains. Both RCTs were conducted at Per Scholas’ original location in the Bronx. Another RCT is underway in other sites to see if the effects generalize to other locations.
  • Disclosure: Arnold Ventures, which funds No-Spin Evidence Review, funded the long-term follow-up of the WorkAdvance Demonstration.

1 Cumulative earnings was a prespecified exploratory (not primary) outcome of the study. We include it in our summary because it captures participants’ overall economic benefit over the 10-year follow-up period.   

2 The graph, copied directly from the 10-year study report, contains a small error: the earnings impact in year 8 should show two asterisks (i.e., statistical significance at the 5 percent level).

3 The study measured earnings using New York state UI records in years 1-5 and NDNH data in years 7-10. Earnings data were missing for two quarters in Year 6 (after the coverage period for the state UI data ended and before the coverage period for the NDNH). To estimate cumulative 10-year earnings impacts, the researchers imputed earnings data for the two missing quarters. Because the researchers did not have access to the same earnings data source for all 10 years, they did not conduct statistical significance tests for cumulative earnings.

4 Because sample members' earnings become more dispersed (i.e., have greater "variance") over time, the confidence interval for the yearly impact estimate widens as the follow-up period lengthens. This makes it harder for the study to detect long-term impacts as statistically significant and can lead to greater variation in the size of yearly impact estimates at the later follow-up points. 

Click or tap a highlight to see No-Spin’s comment

  • Findings

    The abstract accurately summarizes 10-year findings from the four RCTs that comprise the WorkAdvance Demonstration, including the Per Scholas study. Our “No-Spin Study Overview” (below) focuses on the two RCTs - Per Scholas and St. Nicks Alliance – that found more than just short-term earnings impacts.