Formalising Anti-Discrimination Law in Automated Decision Systems

Abstract

Algorithmic discrimination is a critical concern as machine learning models are used in high-stakes decision-making in legally protected contexts. Although substantial research on algorithmic bias and discrimination has led to the development of fairness metrics, several critical legal issues remain unaddressed in practice. The paper addresses three key shortcomings in prevailing ML fairness paradigms: (1) the narrow reliance on prediction or outcome disparity as evidence for discrimination, (2) the lack of nuanced evaluation of estimation error and assumptions that the true causal structure and data-generating process are known, and (3) the overwhelming dominance of US-based analyses which has inadvertently fostered some misconceptions regarding lawful modelling practices in other jurisdictions. To address these gaps, we introduce a novel decision-theoretic framework grounded in anti-discrimination law of the United Kingdom, which has global influence and aligns closely with European and Commonwealth legal systems. We propose the "conditional estimation parity" metric, which accounts for estimation error and the underlying data-generating process, aligning with UK legal standards. We apply our formalism to a real-world algorithmic discrimination case, demonstrating how technical and legal reasoning can be aligned to detect and mitigate unlawful discrimination. Our contributions offer actionable, legally-grounded guidance for ML practitioners, policymakers, and legal scholars seeking to develop non-discriminatory automated decision systems that are legally robust.

References in corpus (16)

• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
• Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment
• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• Fairness in Machine Learning: A Survey
• Strengthening legal protection against discrimination by algorithms and artificial intelligence
• A Clarification of the Nuances in the Fairness Metrics Landscape
• A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices
• Auditing of AI: Legal, Ethical and Technical Approaches
• This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
• What's Sex Got To Do With Fair Machine Learning?
• Transparency, Governance and Regulation of Algorithmic Tools Deployed in the Criminal Justice System: a UK Case Study
• Algorithmic Decision Making with Conditional Fairness
• Fairness through Aleatoric Uncertainty
• Counterfactual Situation Testing: Uncovering Discrimination under Fairness given the Difference
• Hierarchical Bayesian Models to Mitigate Systematic Disparities in Prediction with Proxy Outcomes