COMPAS Recidivism Algorithm Racial Bias (2016)

Incident Summary

In May 2016, ProPublica published an investigation titled “Machine Bias” analyzing the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) algorithm, a proprietary risk assessment tool developed by Northpointe Inc. (now Equivant) and used widely across the United States criminal justice system to predict the likelihood of recidivism.^[1] ProPublica examined COMPAS risk scores assigned to more than 7,000 defendants arrested in Broward County, Florida, between 2013 and 2014, and compared those scores against actual recidivism outcomes over a two-year period.

The analysis found significant racial disparities in the algorithm’s predictions. Black defendants were nearly twice as likely as white defendants to be incorrectly flagged as being at high risk of reoffending (false positives), while white defendants were significantly more likely to be incorrectly labeled as low risk when they did go on to reoffend (false negatives).^[1] Northpointe disputed ProPublica’s conclusions, arguing that the algorithm achieved comparable predictive accuracy across racial groups when measured by a different statistical metric.^[2]

In July 2016, the Wisconsin Supreme Court addressed the use of COMPAS in its ruling in State v. Loomis, holding that the algorithm could be used as one factor in sentencing decisions but that courts must be informed of its limitations, including the fact that the tool was not designed for use in determining the severity of a sentence and that it may exhibit group-level bias.^[3]

Key Facts

• Algorithm: COMPAS (Correctional Offender Management Profiling for Alternative Sanctions), developed by Northpointe Inc. (now Equivant)
• Use: Pretrial risk assessment and sentencing recommendations in U.S. criminal courts
• Investigation scope: Over 7,000 defendants in Broward County, Florida
• Finding: Black defendants nearly twice as likely to be falsely flagged as high-risk compared to white defendants
• Corollary finding: White defendants more likely to be falsely labeled low-risk
• Developer response: Northpointe disputed findings, citing alternative fairness metrics
• Legal ruling: Wisconsin Supreme Court allowed continued use with mandatory disclosure of limitations

Threat Patterns Involved

Primary: Proxy Discrimination — Although COMPAS does not use race as an explicit input variable, ProPublica’s analysis found that the algorithm’s predictions correlated with race in a manner that produced disparate error rates, suggesting that the model relied on features that served as proxies for race.

Secondary: Implicit Authority Transfer — The widespread use of COMPAS scores in sentencing and pretrial decisions effectively transferred significant decision-making authority from judges to an opaque proprietary algorithm, with courts relying on risk scores they could not independently verify or fully understand.

Significance

1. Foundational case in algorithmic fairness. The COMPAS investigation became the most widely cited real-world example of racial bias in algorithmic decision-making and catalyzed an entire field of research into competing definitions of algorithmic fairness.^[1]
2. Impossibility of simultaneous fairness metrics. The subsequent academic debate demonstrated that certain statistical definitions of fairness are mutually exclusive, meaning that an algorithm cannot simultaneously satisfy all reasonable fairness criteria when base rates differ between groups — a finding with profound implications for the use of predictive algorithms in high-stakes decisions.
3. Opacity in high-stakes decisions. COMPAS is a proprietary system whose internal workings are not publicly disclosed, raising fundamental questions about due process when individuals’ liberty is influenced by algorithms they cannot examine or challenge.^[3]
4. Continued use despite documented bias. Despite the investigation’s findings and ongoing legal challenges, COMPAS and similar risk assessment tools remain in use across U.S. courts, illustrating the gap between the identification of algorithmic harms and institutional response.