Randomized Controlled Trial (RCT) of...
New York City’s Summer Youth Employment Program
Reviewed
JPAM (May 2022) published an RCT examining the impacts of New York City’s Summer Youth Employment Program (SYEP) on youth crime. Despite the study abstract’s unambiguously positive portrayal of the results, this high-quality RCT found negligible impacts on crime over a five-year follow-up.
Recent policy discussions have proposed government-guaranteed jobs, including for youth. One key potential benefit of youth employment is a reduction in criminal justice contact. Prior work on summer youth employment programs has documented little-to-no effect of the program on crime during the program but has found decreases in violent and other serious crimes among “at-risk” youth in the year or two after the program. We add to this picture by studying randomized lotteries for access to the New York City Summer Youth Employment Program (SYEP), the largest such program in the United States. We link SYEP data to New York State criminal records data to investigate outcomes of 163,447 youth who participated in a SYEP lottery between 2005 and 2008. We find evidence that SYEP participation decreases arrests and convictions during the program summer, effects that are driven by the small fraction (3 percent) of SYEP youth who are at-risk, as defined by having been arrested before the start of the program. We conclude that an important benefit of SYEPs is the contemporaneous effect during the program summer and that the effect is concentrated among individuals with prior contact with the criminal justice system.
Recent policy discussions have proposed government-guaranteed jobs, including for youth. One key potential benefit of youth employment is a reduction in criminal justice contact. Prior work on summer youth employment programs has documented little-to-no effect of the program on crime during the program but has found decreases in violent and other serious crimes among “at-risk” youth in the year or two after the program. We add to this picture by studying randomized lotteries for access to the New York City Summer Youth Employment Program (SYEP), the largest such program in the United States. We link SYEP data to New York State criminal records data to investigate outcomes of 163,447 youth who participated in a SYEP lottery between 2005 and 2008. We find evidence that SYEP participation decreases arrest and conviction rates by about 0.1 percentage points during the program summer, effects that are driven by the small fraction (3 percent) of SYEP youth who are at-risk, as defined by having been arrested before the start of the program. However, these impacts do not extend beyond the summer, as SYEP and control group youth have similar arrest and conviction rates during each of the 1, 3, and 5 year periods following program completion. We conclude that an important benefit of SYEPs is the contemporaneous effect during the program summer and that the effect is concentrated among individuals with prior contact with the criminal justice system.
No-Spin’s Study Overview
Large, high-quality RCT of New York City’s Summer Youth Employment Program (SYEP) finds negligible impacts on youth crime over a five-year follow-up.
Program and Study Design:
- SYEP, the largest summer youth employment program in the United States, provides New York City youth with 25 hours per week of minimum-wage work for 6-7 weeks during the summer.
- The study sample comprised 163,447 youth ages 16-21 who applied to SYEP in 2005-2008 and were randomly assigned via lottery to a treatment group (which was offered the program) or a control group (which was not).
- 69% of youth in the treatment group (i.e., lottery winners) accepted the offer and participated in SYEP. The study estimated program impacts for these SYEP participants using valid analysis methods (instrumental variables).
- Based on careful review, this was a high-quality RCT (e.g., large sample, baseline balance, negligible sample attrition).
Findings:
- SYEP participation produced statistically significant, but very small, decreases in arrest and conviction rates during the summer program – approximately 0.1 percentage points (arrest rates were 0.64% among SYEP participants vs. 0.77% for controls; conviction rates were 0.17% SYEP vs. 0.24% control).
- However, these impacts did not extend beyond the summer: SYEP and control youth had similar arrest and conviction outcomes during each of the 1, 3, and 5 year periods following program completion. Over the full 5 years: SYEP and control youth had nearly identical arrest rates (11.1%) and conviction rates (3.9%).
Comment:
- This RCT is good example of how studies with very large samples can sometimes find effects that reach statistical significance but are so small as to be of little practical significance.
Click or tap a highlight to see No-Spin’s comment