On the (im)possibility of fairness

Abstract

What does it mean for an algorithm to be fair? Different papers use different notions of algorithmic fairness, and although these appear internally consistent, they also seem mutually incompatible. We present a mathematical setting in which the distinctions in previous papers can be made formal. In addition to characterizing the spaces of inputs (the "observed" space) and outputs (the "decision" space), we introduce the notion of a construct space: a space that captures unobservable, but meaningful variables for the prediction. We show that in order to prove desirable properties of the entire decision-making process, different mechanisms for fairness require different assumptions about the nature of the mapping from construct space to decision space. The results in this paper imply that future treatments of algorithmic fairness should more explicitly state assumptions about the relationship between constructs and observations.

Cited by in corpus (36)

• A Convex Framework for Fair Regression
• Avoiding Discrimination through Causal Reasoning
• Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations
• Meta Balanced Network for Fair Face Recognition
• On the Legal Compatibility of Fairness Definitions
• Two Simple Ways to Learn Individual Fairness Metrics from Data
• FairPrep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing Interventions
• On conditional parity as a notion of non-discrimination in machine learning
• The Authority of "Fair" in Machine Learning
• Quantifying Program Bias
• Decoupled classifiers for fair and efficient machine learning
• Spectral Algorithms for Computing Fair Support Vector Machines
• Counterfactual fairness: removing direct effects through regularization
• Multi-objective Asynchronous Successive Halving
• Fairkit, Fairkit, on the Wall, Who's the Fairest of Them All? Supporting Data Scientists in Training Fair Models
• Fair Division Without Disparate Impact
• Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing
• Transparency, Fairness, Data Protection, Neutrality: Data Management Challenges in the Face of New Regulation
• Teaching Responsible Data Science: Charting New Pedagogical Territory
• Fairness as Equality of Opportunity: Normative Guidance from Political Philosophy
• Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
• On Controllability of AI
• Fairness in Missing Data Imputation
• Where Is the Normative Proof? Assumptions and Contradictions in ML Fairness Research
• RAWLSNET: Altering Bayesian Networks to Encode Rawlsian Fair Equality of Opportunity
• Beyond traditional assumptions in fair machine learning
• Machine Learning in Population and Public Health
• Everything is Relative: Understanding Fairness with Optimal Transport
• Fair for All: Best-effort Fairness Guarantees for Classification
• Fairness-Aware Online Personalization
• OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning
• Towards Data-Driven Affirmative Action Policies under Uncertainty
• How Fair is Fairness-aware Representative Ranking and Methods for Fair Ranking
• Social Responsibility of Algorithms
• Continuous Subject-in-the-Loop Integration: Centering AI on Marginalized Communities
• Evaluating the fairness of fine-tuning strategies in self-supervised learning