The Measure and Mismeasure of Fairness

Abstract

The field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last decade, several formal, mathematical definitions of fairness have gained prominence. Here we first assemble and categorize these definitions into two broad families: (1) those that constrain the effects of decisions on disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions typically result in strongly Pareto dominated decision policies. For example, in the case of college admissions, adhering to popular formal conceptions of fairness would simultaneously result in lower student-body diversity and a less academically prepared class, relative to what one could achieve by explicitly tailoring admissions policies to achieve desired outcomes. In this sense, requiring that these fairness definitions hold can, perversely, harm the very groups they were designed to protect. In contrast to axiomatic notions of fairness, we argue that the equitable design of algorithms requires grappling with their context-specific consequences, akin to the equitable design of policy. We conclude by listing several open challenges in fair machine learning and offering strategies to ensure algorithms are better aligned with policy goals.

References in corpus (12)

• Towards A Rigorous Science of Interpretable Machine Learning
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Equality of Opportunity in Supervised Learning
• Counterfactual Fairness
• On Fairness and Calibration
• Fairness Without Demographics in Repeated Loss Minimization
• A Short-term Intervention for Long-term Fairness in the Labor Market
• Censoring Representations with an Adversary
• Exposing the Probabilistic Causal Structure of Discrimination
• Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment
• Bias In, Bias Out? Evaluating the Folk Wisdom
• Algorithmic Decision Making in the Presence of Unmeasured Confounding

Cited by in corpus (126)

• On the Opportunities and Risks of Foundation Models
• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• Fairness in Machine Learning: A Survey
• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• Towards a Critical Race Methodology in Algorithmic Fairness
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction
• Machine Learning Testing: Survey, Landscapes and Horizons
• Aligning AI With Shared Human Values
• Conservative AI and social inequality: Conceptualizing alternatives to bias through social theory
• A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
• Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services
• What's Sex Got To Do With Fair Machine Learning?
• Fair Regression: Quantitative Definitions and Reduction-based Algorithms
• Fair Regression for Health Care Spending
• Machine learning fairness notions: Bridging the gap with real-world applications
• Training individually fair ML models with Sensitive Subspace Robustness
• Fairness Score and Process Standardization: Framework for Fairness Certification in Artificial Intelligence Systems
• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric
• Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination
• On the Legal Compatibility of Fairness Definitions
• Not one but many Tradeoffs: Privacy Vs. Utility in Differentially Private Machine Learning
• Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference
• Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
• Fairness Evaluation in Presence of Biased Noisy Labels
• Can Explainable AI Explain Unfairness? A Framework for Evaluating Explainable AI
• Improving the Fairness of Deep Generative Models without Retraining
• Bias in Data-driven AI Systems -- An Introductory Survey
• Avoiding Disparity Amplification under Different Worldviews
• Fair Allocation through Selective Information Acquisition
• "What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairness
• Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity
• A Simple, Statistically Robust Test of Discrimination
• DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems
• Enforcing Group Fairness in Algorithmic Decision Making: Utility Maximization Under Sufficiency
• Cold Start Similar Artists Ranking with Gravity-Inspired Graph Autoencoders
• Equalized odds postprocessing under imperfect group information
• Conditional Learning of Fair Representations
• FairCanary: Rapid Continuous Explainable Fairness
• Detecting disparities in police deployments using dashcam data
• What does it mean to solve the problem of discrimination in hiring? Social, technical and legal perspectives from the UK on automated hiring systems
• Questioning the assumptions behind fairness solutions
• Empirical Analysis of Multi-Task Learning for Reducing Model Bias in Toxic Comment Detection
• Axes for Sociotechnical Inquiry in AI Research
• Understanding Agent Incentives using Causal Influence Diagrams. Part I: Single Action Settings
• Removing the influence of a group variable in high-dimensional predictive modelling
• Advertising for Demographically Fair Outcomes
• Geographic ratemaking with spatial embeddings
• Markovian Score Climbing: Variational Inference with KL(p||q)
• Crowdsourcing Impacts: Exploring the Utility of Crowds for Anticipating Societal Impacts of Algorithmic Decision Making
• Is calibration a fairness requirement? An argument from the point of view of moral philosophy and decision theory
• Bias, Fairness, and Accountability with AI and ML Algorithms
• Interpretable and Fair Boolean Rule Sets via Column Generation
• Ethical Adversaries: Towards Mitigating Unfairness with Adversarial Machine Learning
• Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systems
• From the Fair Distribution of Predictions to the Fair Distribution of Social Goods: Evaluating the Impact of Fair Machine Learning on Long-Term Unemployment
• Deontological Ethics By Monotonicity Shape Constraints
• Predicting Early Dropout: Calibration and Algorithmic Fairness Considerations
• Dynamic fairness - Breaking vicious cycles in automatic decision making
• Characterizing Intersectional Group Fairness with Worst-Case Comparisons
• Principal Fairness for Human and Algorithmic Decision-Making
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Gender Slopes: Counterfactual Fairness for Computer Vision Models by Attribute Manipulation
• Assessing Fairness in Classification Parity of Machine Learning Models in Healthcare
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges
• Active Fairness Instead of Unawareness
• What will it take to generate fairness-preserving explanations?
• Does enforcing fairness mitigate biases caused by subpopulation shift?
• A non-discriminatory approach to ethical deep learning
• Hypergraph Clustering for Finding Diverse and Experienced Groups
• Metric-Free Individual Fairness in Online Learning
• With Malice Towards None: Assessing Uncertainty via Equalized Coverage
• Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Conformal Prediction Sets
• Unintended Selection: Persistent Qualification Rate Disparities and Interventions
• Challenging common interpretability assumptions in feature attribution explanations
• ProPublica's COMPAS Data Revisited
• Group Fairness in Bandit Arm Selection
• (Un)fairness in Post-operative Complication Prediction Models
• A Possibility in Algorithmic Fairness: Can Calibration and Equal Error Rates Be Reconciled?
• Towards Threshold Invariant Fair Classification
• Hierarchical Bayesian Models to Mitigate Systematic Disparities in Prediction with Proxy Outcomes
• Adversarial Scrutiny of Evidentiary Statistical Software
• Cohort Shapley value for algorithmic fairness
• Fairness criteria through the lens of directed acyclic graphical models
• Learning by Design: Structuring and Documenting the Human Choices in Machine Learning Development
• Algorithmic and Economic Perspectives on Fairness
• Anticipatory Ethics and the Role of Uncertainty
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Assessing Disparate Impacts of Personalized Interventions: Identifiability and Bounds
• Quantifying Infra-Marginality and Its Trade-off with Group Fairness
• Leveraging Administrative Data for Bias Audits: Assessing Disparate Coverage with Mobility Data for COVID-19 Policy
• Dropping Standardized Testing for Admissions Trades Off Information and Access
• On the Impact of Data Quality on Image Classification Fairness
• The Challenge of Value Alignment: from Fairer Algorithms to AI Safety
• Fair Decision Rules for Binary Classification
• Investigating Trade-offs in Utility, Fairness and Differential Privacy in Neural Networks
• Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning
• AI and Holistic Review: Informing Human Reading in College Admissions
• A Fully Bayesian, Logistic Regression Tracking Algorithm for Mitigating Disparate Misclassification
• Towards the Right Kind of Fairness in AI
• Artificial Tikkun Olam: AI Can Be Our Best Friend in Building an Open Human-Computer Society
• Mimetic vs Anchored Value Alignment in Artificial Intelligence
• Exploring the Unfairness of DP-SGD Across Settings
• Getting Fairness Right: Towards a Toolbox for Practitioners
• Equality of opportunity in travel behavior prediction with deep neural networks and discrete choice models
• A Distributed Fair Machine Learning Framework with Private Demographic Data Protection
• A Mulching Proposal
• Randomized Classifiers vs Human Decision-Makers: Trustworthy AI May Have to Act Randomly and Society Seems to Accept This
• Everything is Relative: Understanding Fairness with Optimal Transport
• Hidden Technical Debts for Fair Machine Learning in Financial Services
• Addressing Fairness, Bias and Class Imbalance in Machine Learning: the FBI-loss
• Explaining how your AI system is fair
• Fairness for Whom? Critically reframing fairness with Nash Welfare Product
• Measuring a Texts Fairness Dimensions Using Machine Learning Based on Social Psychological Factors
• The Limits of Computation in Solving Equity Trade-Offs in Machine Learning and Justice System Risk Assessment
• Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation
• Human-Imitating Metrics for Training and Evaluating Privacy Preserving Emotion Recognition Models Using Sociolinguistic Knowledge
• On the Fairness of Machine-Assisted Human Decisions
• Two-step adversarial debiasing with partial learning -- medical image case-studies
• When black box algorithms are (not) appropriate: a principled prediction-problem ontology
• The 2021 RecSys Challenge Dataset: Fairness is not optional
• Stateful Strategic Regression
• Local Justice and the Algorithmic Allocation of Societal Resources
• Detecting and Reducing Bias in a High Stakes Domain