On the (im)possibility of fairness

Abstract

What does it mean for an algorithm to be fair? Different papers use different notions of algorithmic fairness, and although these appear internally consistent, they also seem mutually incompatible. We present a mathematical setting in which the distinctions in previous papers can be made formal. In addition to characterizing the spaces of inputs (the "observed" space) and outputs (the "decision" space), we introduce the notion of a construct space: a space that captures unobservable, but meaningful variables for the prediction. We show that in order to prove desirable properties of the entire decision-making process, different mechanisms for fairness require different assumptions about the nature of the mapping from construct space to decision space. The results in this paper imply that future treatments of algorithmic fairness should more explicitly state assumptions about the relationship between constructs and observations.

References in corpus (1)

• A survey on measuring indirect discrimination in machine learning

Cited by in corpus (81)

• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• FA*IR: A Fair Top-k Ranking Algorithm
• Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
• Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
• The Frontiers of Fairness in Machine Learning
• The Ethics of AI in Education
• A Clarification of the Nuances in the Fairness Metrics Landscape
• A Convex Framework for Fair Regression
• Avoiding Discrimination through Causal Reasoning
• An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction
• Designing Fair Ranking Schemes
• A Short-term Intervention for Long-term Fairness in the Labor Market
• FairSight: Visual Analytics for Fairness in Decision Making
• The Butterfly Effect in Artificial Intelligence Systems: Implications for AI Bias and Fairness
• Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations
• Fairness in Ranking: A Survey
• Conservative AI and social inequality: Conceptualizing alternatives to bias through social theory
• Operationalizing Individual Fairness with Pairwise Fair Representations
• A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
• Online Learning with an Unknown Fairness Metric
• Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction
• Meta Balanced Network for Fair Face Recognition
• Machine learning fairness notions: Bridging the gap with real-world applications
• On the Moral Justification of Statistical Parity
• Optimized Data Pre-Processing for Discrimination Prevention
• It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks
• Improving Fairness via Federated Learning
• Blind Justice: Fairness with Encrypted Sensitive Attributes
• Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
• On the Legal Compatibility of Fairness Definitions
• What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective
• Two Simple Ways to Learn Individual Fairness Metrics from Data
• Avoiding Disparity Amplification under Different Worldviews
• FairPrep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing Interventions
• On conditional parity as a notion of non-discrimination in machine learning
• Enforcing Group Fairness in Algorithmic Decision Making: Utility Maximization Under Sufficiency
• The Authority of "Fair" in Machine Learning
• My Model is Unfair, Do People Even Care? Visual Design Affects Trust and Perceived Bias in Machine Learning
• FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
• Quantifying Program Bias
• Label-Imbalanced and Group-Sensitive Classification under Overparameterization
• Spectral Algorithms for Computing Fair Support Vector Machines
• Decoupled classifiers for fair and efficient machine learning
• Multi-objective Asynchronous Successive Halving
• Counterfactual fairness: removing direct effects through regularization
• Principal Fairness for Human and Algorithmic Decision-Making
• Fairness in Reinforcement Learning
• Fairkit, Fairkit, on the Wall, Who's the Fairest of Them All? Supporting Data Scientists in Training Fair Models
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing
• Transparency, Fairness, Data Protection, Neutrality: Data Management Challenges in the Face of New Regulation
• Fair Division Without Disparate Impact
• Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting
• Debiasing representations by removing unwanted variation due to protected attributes
• On Controllability of AI
• Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
• Teaching Responsible Data Science: Charting New Pedagogical Territory
• Fairness as Equality of Opportunity: Normative Guidance from Political Philosophy
• Fairness in Missing Data Imputation
• Where Is the Normative Proof? Assumptions and Contradictions in ML Fairness Research
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Fairness Through Causal Awareness: Learning Latent-Variable Models for Biased Data
• RAWLSNET: Altering Bayesian Networks to Encode Rawlsian Fair Equality of Opportunity
• Towards the Right Kind of Fairness in AI
• Getting Fairness Right: Towards a Toolbox for Practitioners
• Fair for All: Best-effort Fairness Guarantees for Classification
• Beyond traditional assumptions in fair machine learning
• Everything is Relative: Understanding Fairness with Optimal Transport
• OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning
• Fairness-Aware Online Personalization
• Machine Learning in Population and Public Health
• Social Responsibility of Algorithms
• Pairwise Fairness for Ordinal Regression
• Towards Data-Driven Affirmative Action Policies under Uncertainty
• Continuous Subject-in-the-Loop Integration: Centering AI on Marginalized Communities
• Towards a Flexible Framework for Algorithmic Fairness
• Evaluating the fairness of fine-tuning strategies in self-supervised learning
• Explaining how your AI system is fair
• Access to Population-Level Signaling as a Source of Inequality
• How Fair is Fairness-aware Representative Ranking and Methods for Fair Ranking