Randomized Controlled Trial (RCT) of the...

Building Assets and Reducing Risks (BARR) educational system, as delivered in 9th grade

Reviewed

K-12 Education
Somewhat accurately reported
Credible positive finding

AIR (August 2022) published an RCT of Building Assets and Reducing Risks (BARR) – an educational system aimed at improving staff-to-staff, student-to-staff, and student-to-student relationships –  as delivered in 9th grade. This generally high-quality RCT found positive impacts on 9th grade outcomes, such as a 6 percentage point increase in the proportion of students passing all their core classes, but additional research is needed to determine if effects persist into later grades.

Relevant Excerpt from the Online Study Overview:

Notable Scale-up Evaluation Findings

  • The BARR approach improved teachers’ collaboration with their peers, their data use, and a range of other teacher outcomes. Among teacher outcomes, the most substantial effects were on teacher collaboration with and view of colleagues, teacher use of data, and view of school supports. Positive effects on teacher-reported outcomes persisted even after schools operated virtually for much of the 2019–20 school year due to the pandemic.
  • The BARR approach had substantial and statistically significant impacts on the proportion of students who passed all their core courses. For example, 80 percent of ninth graders in the treatment group passed all their core courses, compared to 74 percent of students enrolled in control group schools that operated “business as usual.” These effects were strongest for male students, students of color, students eligible for free or reduced-price lunch, English learners, and students with disabilities.
  • BARR significantly reduced chronic absenteeism. The rate of chronic absenteeism among ninth graders in the treatment group was 19 percent, compared to 22 percent in “business as usual” schools. Positive effects on chronic absenteeism were strongest for male students and students of color.
  • There is some evidence that BARR had a positive impact on Preliminary Scholastic Aptitude Test (PSAT) scores. However, the PSAT test could not be administered in 2020 due to the pandemic so these results were available only for the first two cohorts.

The scale-up evaluation findings are consistent with evidence from previous studies which concluded that BARR is an effective model for schools that want to reduce course failure and chronic absenteeism. The collective body of evidence on the BARR model also showed that students and teachers using the BARR approach reported better school experiences, including stronger relationships among teachers, among students, and between teachers and students.

Full Study Report

Notable Scale-up Evaluation Findings

  • The BARR approach improved teachers’ collaboration with their peers, their data use, and a range of other teacher outcomes. Among teacher outcomes, the most substantial effects were on teacher collaboration with and view of colleagues, teacher use of data, and view of school supports. Positive effects on teacher-reported outcomes persisted even after schools operated virtually for much of the 2019–20 school year due to the pandemic.
  • The BARR approach had substantial and statistically significant impacts on the proportion of students who passed all their core courses. For example, 80 percent of ninth graders in the treatment group passed all their core courses, compared to 74 percent of students enrolled in control group schools that operated “business as usual.” These effects were strongest for male students, students of color, students eligible for free or reduced-price lunch, English learners, and students with disabilities.
  • BARR significantly reduced chronic absenteeism. The rate of chronic absenteeism among ninth graders in the treatment group was 19 percent, compared to 22 percent in “business as usual” schools. Positive effects on chronic absenteeism were strongest for male students and students of color.
  • There is some evidence that BARR had a positive impact on Preliminary Scholastic Aptitude Test (PSAT) scores,  but it is only suggestive due to high sample attrition. Additionally,  However, the PSAT test could not be administered in 2020 due to the pandemic so these results were available only for the first two cohorts.

The scale-up evaluation findings are consistent with evidence from previous studies which concluded that BARR is an effective model for schools that want to reduce course failure and chronic absenteeism. The collective body of evidence on the BARR model also showed that students and teachers using the BARR approach reported better school experiences, including stronger relationships among teachers, among students, and between teachers and students.  Longer-term studies of BARR would be valuable to determine whether the positive impacts found in 9th grade persist into subsequent grades.

No-Spin’s Study Overview

High-quality RCT of the Building Assets and Reducing Risks (BARR) educational system, as delivered in 9th grade, finds positive impacts on end of year outcomes, such as a 6 percentage point increase in the proportion of students passing all their core classes. Additional research is needed to determine if effects persist into later grades.

Program:

  • As previously described, “BARR works with schools to improve staff-to-staff, student-to-staff, and student-to-student relationships. BARR brings together core subject teachers serving the same students to discuss the progress and challenges of all their students. Drawing on personal relationships, teachers discuss how to use each student’s individual strengths to overcome challenges and improve their in-school experience. Students receive lessons from their core teachers to develop social and emotional skills.”
  • This study evaluated BARR in 9th grade, where BARR aims to facilitate the challenging transition from middle to high school.

Study Design:

  • The study randomly assigned 66 high schools across 12 states and the District of Columbia to either implement BARR (treatment group, 37 schools with 12,122 9th graders) or continue services as usual (control group, 29 schools with 9,407 9th graders).
  • Based on our careful review, this was a generally high-quality RCT (e.g., large sample, baseline balance, prespecified outcomes, valid analyses).

Findings:

  • The study found a modest, statistically significant effect on its prespecified primary outcome related to progression through 9th grade: an increase in the proportion of core course credits earned (the treatment group earned an average of 89.1% vs. 85.5% for the control group, p<0.01).
  • The study also found statistically significant or near-significant effects on related exploratory outcomes in 9th grade, including likelihood of passing all core classes (80% treatment vs. 74% control, p<0.01), GPA (2.6 treatment vs. 2.5 control, p<0.01), rate of chronic absenteeism (19% treatment vs. 22% control, p<0.05), and persistence to 10th grade (88% treatment vs. 85% control, p<0.10). The study found no impact on the rate of student suspensions.
  • The study found small positive effects on its two other prespecified primary outcomes, measured in the fall of 10th grade: PSAT math scores (effect size = 0.07, p<0.05) and PSAT reading scores (effect size = 0.07, p=.11, not statistically significant). However, these findings are only suggestive – not reliable – due to high rates of sample loss that differed between the treatment group (32%) and control group (43%), undermining the initial randomization.

Comment:

  • The chief limitation in BARR’s evidence base, including this and two prior 9th grade RCTs (here and here), is lack of reliable follow-up beyond the 9th grade year to determine whether the program’s positive impacts persist or fade in subsequent grades.  

Click or tap a highlight to see No-Spin’s comment

Related Reviews