Fairness

Definition

Fairness in machine learning refers to the requirement that AI systems treat individuals and demographic groups equitably, avoiding unjustified disparities in outcomes, error rates, or treatment. The concept encompasses numerous formal mathematical definitions — including demographic parity, equalised odds, predictive parity, and individual fairness — which have been proven to be mutually incompatible in most non-trivial settings. This incompatibility, known as the impossibility theorem of fairness, means that practitioners must make value-laden choices about which fairness criteria to prioritise in a given context. Fairness is therefore understood as a sociotechnical challenge rather than a purely computational one, requiring input from affected communities, domain experts, ethicists, and legal scholars alongside technical implementation.

How It Relates to AI Threats

Fairness is a central governance concern spanning the Discrimination & Social Harm and Human-AI Control domains. When AI systems lack adequate fairness constraints, they can perpetuate or amplify historical patterns of discrimination through proxy discrimination, allocational harm, and data imbalance bias. The multiplicity of fairness definitions creates additional risk: an AI system may satisfy one fairness criterion while violating another, allowing deployers to claim compliance while discriminatory outcomes persist. The absence of consensus on fairness standards also complicates regulatory enforcement, as different jurisdictions and application domains may apply different fairness requirements to the same type of system.

Why It Occurs

• Multiple mathematically valid fairness definitions are provably incompatible, requiring value-laden trade-offs
• Training data reflects historical inequities that models learn and reproduce without explicit fairness constraints
• Fairness testing requires demographic data that organisations may lack or be legally restricted from collecting
• Technical fairness interventions can reduce model accuracy, creating tension with performance-oriented objectives
• Organisational incentives often prioritise deployment speed over comprehensive fairness evaluation

Real-World Context

Fairness concerns are documented in incidents involving credit scoring (INC-13-0001) and hiring algorithms (INC-18-0002), where systems produced disparate outcomes across demographic groups. The EU AI Act requires providers of high-risk AI systems to implement bias testing and mitigation measures. The U.S. White House Blueprint for an AI Bill of Rights identifies protection from algorithmic discrimination as a core principle. Academic contributions from Chouldechova, Kleinberg, and others have formalised the mathematical constraints on simultaneous fairness, informing both technical practice and regulatory design.