ASSISTments, a Low-Cost Online Tool to Support 7th Grade Math Learning

No-Spin’s Study Overview

High-quality RCT of ASSISTments – a low-cost online tool to support 7th grade math learning – finds a sizable gain (~31% of a grade-level)¹ in math scores on the state test at 2 year follow-up.

Program:

• ASSISTments is an online study tool that assists 7th grade students with math independent practice (e.g., homework) and produces real-time reports for teachers on student performance. The program’s cost is less than $100 per student.

Study Design:

• The study sample comprised 63 public schools in North Carolina with 9,703 7th graders (61% low income). The schools were randomized to ASSISTments vs control. Based on careful review, this was a high-quality RCT (e.g., baseline balance, modest attrition, prespecified analyses).
  

Findings:​

• The study couldn't measure end-of-year effects due to COVID, but at end of 8th grade (1 year post-program), it found a statistically significant impact on state test scores in math: Effect size 0.10, or about a 31% improvement over the expected annual gain in math for 8th graders.¹
• The study also found suggestive evidence (near statistically significant) that ASSISTments produced larger gains for students with higher levels of prior math achievement than students with lower levels of prior achievement.

Comment:​

• This is a positive replication of a prior high-quality RCT of ASSISTments in Maine (effect size 0.18 at end of 7th grade) and shows that the effects endure after program completion. (Here’s an evidence summary that draws on both studies.)
• Taken together, the Maine and North Carolina studies constitute strong, replicated RCT evidence of effectiveness – providing confidence that schools which adopt and faithfully implement ASSISTments will see meaningful gains in student math achievement.
• The suggestive finding in the North Carolina study of a greater impact for higher-performing versus lower-performing students contradicts an earlier finding from the Maine study of greater impact for lower-performing students. The question of which group benefits more would be valuable to examine in future studies.

• Arnold Ventures, which funds No-Spin Evidence Review, helped fund the North Carolina ASSISTments RCT.

   

  ---

  ¹ _{The average annual gain in math achievement for U.S. eighth grade students on six nationally normed tests is 0.32 standard deviations (see Bloom, Hill, Black, and Lipsey, 2007). ASSISTments’ impact of 0.10 represents a 31% improvement over this annual gain. One caveat regarding this comparison is that while the North Carolina state test used to estimate ASSISTments’ impact was similar to the six tests in Bloom 2007 in measuring broadband math achievement (versus just a specific topic such as fractions), it was – unlike the six tests – not nationally normed. For that reason, the comparison of ASSISTments’ impact against the Bloom 2007 benchmark is best viewed as illustrative rather than exact.}