Objective: Pay-for-performance (P4P) initiatives hold promise for improving health care delivery but are rarely applied to behavioral health or tested in randomized controlled trials (RCTs). This RCT examined the effectiveness of a P4P initiative to reduce total cost of 24-hour care among patients with high needs for psychiatric care in a large county in California.
Methods: From August 2016 to March 2022, a total of 652 adult residents of Santa Clara County, California, were enrolled in a P4P initiative (mean±SD age=46.7±13.3 years, 61% male, 51% White, and 60% diagnosed as having a bipolar or psychotic disorder). Participants were randomly assigned to usual full-service partnerships from the county (N=327) or a comparable level of care from a contractor who agreed to a schedule of financial penalties and rewards based on whether enrollees (N=325) used more or less care than a historical cohort of similar county patients. The primary outcome was total cost of 24-hour psychiatric services. Secondary outcomes were costs of each of the 24-hour care services.
Conclusions: A P4P initiative for high-need patients was no more effective than usual care for reducing costs of 24-hour psychiatric care.
We have no suggested revisions to the study abstract.
No-Spin’s Study Overview
High-quality RCT of a county government’s Pay for Performance (P4P) approach to delivering care to adults with serious mental illness, aimed at reducing the use and costs of 24-hour psychiatric services, finds no significant effect on the per-patient costs of such services over three years.
Program and Study Design:
- The study sample comprised 652 adults in Santa Clara County, CA with serious mental illness and a history of high utilization of psychiatric services. In the 12 months prior to study entry, sample members’ average cost of 24-hour psychiatric services was $103,654.
- Sample members were randomly assigned to a P4P group (N=325) or a control group (N=327).
- Both groups received similar wraparound supports, including intensive case management and housing subsidies. In the P4P group, these supports were delivered by a contracted provider that received financial rewards (or penalties) if patients’ use of 24-hour services fell below (or exceeded) performance targets. In the control group, these wraparound supports were delivered by the County’s standard (“full-service partnership”) providers without P4P financial incentives.
- Based on our review criteria, this was a mostly well-conducted RCT.
Findings:
- The study found no significant effects of P4P on the total per-patient cost of 24-hour psychiatric services in any of the three years following study entry (see graph below, based on the report’s Table 3).
- As shown in the graph, costs declined by more than 50% in both the P4P group and the control group over the three-year follow-up period.
Comment:
- The findings illustrate why a control group is essential to determining whether a program works. A simple before-and-after comparison of costs in the P4P group would have incorrectly found that P4P generated substantial savings. The fact that costs decreased just as much in the control group shows that the decline cannot be attributed to P4P. Rather, the large cost decline in both groups likely reflects “regression to the mean” among patients selected for their unusually high prior use of psychiatric services.
- Sizable, parallel changes in outcomes for the treatment and control group are common in RCTs, which is why studies without a control group can often produce erroneous findings.
- A study limitation is that the cost comparison between the P4P and control groups excluded wraparound supports (case management and housing services). Including these costs would have provided a more complete comparison, although doing so may not have materially changed the findings since both groups received similar supports.
Click or tap a highlight to see No-Spin’s comment