Big Brothers Big Sisters (BBBS) Community-Based Mentoring for At-Risk Youth

No-Spin’s Study Overview

A national RCT of Big Brothers Big Sisters (BBBS) Community-Based Mentoring for at-risk youth finds highly promising, but not definitive, evidence of sizable reductions (20-40%) in delinquency and substance use at the four-year follow-up.

Program:

• BBBS of America is the largest mentoring organization in the United States. Its Community-Based Mentoring program matches at-risk youth with adult volunteer mentors from the surrounding community. Mentors and youth are expected to get together regularly for at least one year to engage in activities of their own choosing. The program cost is approximately $4,600 per youth.​

Study Design:

• The study randomly assigned 1,358 low-income youths at 17 BBBS agencies nationwide to (i) a treatment group, which BBBS sought to match with a mentor; or (ii) a control group that was eligible for the program after the four-year study period.
• The youths’ average age was 12, 69% were Black or Hispanic, 63% were male, and their families’ average income was about $30,000 per year.
• 68% of treatment group youths were matched with a mentor over the four years after study entry, and the average match lasted 22.5 months.
• Based on careful review, this was a high-quality RCT in many respects (e.g., baseline balance in both the original sample and four-year follow-up sample, preregistered outcomes, valid analyses). Study limitations are noted below.

Findings:

• The study found sizable impacts on its four preregistered primary outcomes – all related to delinquency and substance use – at follow-up four years after study entry. The impacts are shown in the following table (adapted from Table 4 on page 38 of the posted study report):

Comment:

We believe these findings are highly promising but not definitive for three reasons. 

• First, the three findings that are based on youth and/or parent reports – property-related and violence-related delinquent behavior, and substance use – suffered from sample loss (“attrition”) that differed between the treatment and control groups (25% treatment vs. 16% control for delinquency, and 30% treatment vs. 21% control for substance use). This creates risk of bias under established standards (WWC) by partly undoing the initial randomization.
• Second, the three youth and/or parent-reported findings are potentially vulnerable to “social-desirability bias” – i.e., the treatment group’s reporting overly positive outcomes out of gratitude to the mentor or program.
• Third, the finding that wasn’t at risk of attrition or social desirability bias – lower arrest rate in the treatment vs. control group based on administrative records – wasn’t statistically significant, and is therefore suggestive but not reliable.

Disclosure: Arnold Ventures, which funds No-Spin Evidence Review, funded this RCT.