On Fairness and Calibration

Abstract

The machine learning community has become increasingly concerned with the potential for bias and discrimination in predictive models. This has motivated a growing line of work on what it means for a classification procedure to be "fair." In this paper, we investigate the tension between minimizing error disparity across different population groups while maintaining calibrated probability estimates. We show that calibration is compatible only with a single error constraint (i.e. equal false-negatives rates across groups), and show that any algorithm that satisfies this relaxation is no better than randomizing a percentage of predictions for an existing classifier. These unsettling findings, which extend and generalize existing results, are empirically confirmed on several datasets.

First two authors contributed equally. NIPS 2017

Cited by in corpus (125)

• Fairness in Machine Learning: A Survey
• The Measure and Mismeasure of Fairness
• Fairness in Machine Learning
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
• Fairness in Credit Scoring: Assessment, Implementation and Profit Implications
• Learning Adversarially Fair and Transferable Representations
• An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction
• Identifying and Correcting Label Bias in Machine Learning
• AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wild
• Does Object Recognition Work for Everyone?
• Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness
• Empirical Risk Minimization under Fairness Constraints
• Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
• Calibration for the (Computationally-Identifiable) Masses
• The Role of Cooperation in Responsible AI Development
• Mitigating Bias in Federated Learning
• Mitigating Gender Bias in Natural Language Processing: Literature Review
• Fairness for Robust Log Loss Classification
• Calibration of Encoder Decoder Models for Neural Machine Translation
• FairBatch: Batch Selection for Model Fairness
• Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• Improved Adversarial Learning for Fair Classification
• Inherent Tradeoffs in Learning Fair Representations
• Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
• Demographics and discussion influence views on algorithmic fairness
• End-To-End Bias Mitigation: Removing Gender Bias in Deep Learning
• FACT: A Diagnostic for Group Fairness Trade-offs
• Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
• FairPrep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing Interventions
• Explainability for fair machine learning
• Fairness in Online Social Network Timelines: Measurements, Models and Mechanism Design
• A Distributionally Robust Approach to Fair Classification
• Training Fair Models in Federated Learning without Data Privacy Infringement
• Achieving Fairness in the Stochastic Multi-armed Bandit Problem
• Ensuring Fairness Beyond the Training Data
• Auditing and Achieving Intersectional Fairness in Classification Problems
• Equalized odds postprocessing under imperfect group information
• Equalizing Recourse across Groups
• Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning
• Fairness-aware Classification: Criterion, Convexity, and Bounds
• Questioning the assumptions behind fairness solutions
• Multi-Objective Automatic Machine Learning with AutoxgboostMC
• BeFair: Addressing Fairness in the Banking Sector
• Recovering from Biased Data: Can Fairness Constraints Improve Accuracy?
• Learning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness Constraints
• Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information
• FR-Train: A Mutual Information-Based Approach to Fair and Robust Training
• FNNC: Achieving Fairness through Neural Networks
• Characterizing Fairness Over the Set of Good Models Under Selective Labels
• Causal Interventions for Fairness
• Fairness Assessment for Artificial Intelligence in Financial Industry
• Reducing the Teacher-Student Gap via Spherical Knowledge Disitllation
• Removing biased data to improve fairness and accuracy
• On Calibration and Out-of-domain Generalization
• Bias, Fairness, and Accountability with AI and ML Algorithms
• Fairness With Minimal Harm: A Pareto-Optimal Approach For Healthcare
• Modeling Techniques for Machine Learning Fairness: A Survey
• Predicting Early Dropout: Calibration and Algorithmic Fairness Considerations
• Fair Machine Learning Under Partial Compliance
• Biases in Data Science Lifecycle
• Sample Complexity of Uniform Convergence for Multicalibration
• Soft Calibration Objectives for Neural Networks
• Online Abuse of UK MPs from 2015 to 2019: Working Paper
• Counterfactual fairness: removing direct effects through regularization
• Multi-objective Asynchronous Successive Halving
• Fair Classification with Noisy Protected Attributes: A Framework with Provable Guarantees
• Approaching Machine Learning Fairness through Adversarial Network
• Diversity regularization in deep ensembles
• Addressing multiple metrics of group fairness in data-driven decision making
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• On the Role of Dataset Quality and Heterogeneity in Model Confidence
• Statistical Equity: A Fairness Classification Objective
• Detection and Mitigation of Bias in Ted Talk Ratings
• Fair Sequential Selection Using Supervised Learning Models
• Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing
• Towards Self-Regulating AI: Challenges and Opportunities of AI Model Governance in Financial Services
• Keeping Designers in the Loop: Communicating Inherent Algorithmic Trade-offs Across Multiple Objectives
• Capturing Label Distribution: A Case Study in NLI
• Dynamic Modeling and Equilibria in Fair Decision Making
• Unbiased Subdata Selection for Fair Classification: A Unified Framework and Scalable Algorithms
• Near Neighbor: Who is the Fairest of Them All?
• Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration
• Augmented Fairness: An Interpretable Model Augmenting Decision-Makers' Fairness
• A Possibility in Algorithmic Fairness: Can Calibration and Equal Error Rates Be Reconciled?
• Fair Classification and Social Welfare
• Towards Threshold Invariant Fair Classification
• Race and Religion in Online Abuse towards UK Politicians: Working Paper
• FairMask: Better Fairness via Model-based Rebalancing of Protected Attributes
• Metrics and methods for a systematic comparison of fairness-aware machine learning algorithms
• Online Multivalid Learning: Means, Moments, and Prediction Intervals
• Priority-based Post-Processing Bias Mitigation for Individual and Group Fairness
• FairCal: Fairness Calibration for Face Verification
• Fairness criteria through the lens of directed acyclic graphical models
• Beyond Individual and Group Fairness
• Non-Parametric Calibration for Classification
• An optimizable scalar objective value cannot be objective and should not be the sole objective
• Algorithm Fairness in AI for Medicine and Healthcare
• Learning Fairness-aware Relational Structures
• Stronger Calibration Lower Bounds via Sidestepping
• Biased Models Have Biased Explanations
• Fairness in Ranking under Uncertainty
• Harnessing value from data science in business: ensuring explainability and fairness of solutions
• Adaptation and Generalization for Unknown Sensitive Factors of Variations
• Fair Classification with Adversarial Perturbations
• Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information
• Explainability-aided Domain Generalization for Image Classification
• The Sharpe predictor for fairness in machine learning
• Fair Sparse Regression with Clustering: An Invex Relaxation for a Combinatorial Problem
• Beyond traditional assumptions in fair machine learning
• Towards Integrating Fairness Transparently in Industrial Applications
• Right Decisions from Wrong Predictions: A Mechanism Design Alternative to Individual Calibration
• Knowing More About Questions Can Help: Improving Calibration in Question Answering
• Characterizing the risk of fairwashing
• Fairness Preferences, Actual and Hypothetical: A Study of Crowdworker Incentives
• Local Justice and the Algorithmic Allocation of Societal Resources
• Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems
• The Limits of Computation in Solving Equity Trade-Offs in Machine Learning and Justice System Risk Assessment
• Moral Decision-Making in Medical Hybrid Intelligent Systems: A Team Design Patterns Approach to the Bias Mitigation and Data Sharing Design Problems
• Coping with Mistreatment in Fair Algorithms
• Selective Regression Under Fairness Criteria
• Fairness for Whom? Critically reframing fairness with Nash Welfare Product
• Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance
• A Sociotechnical View of Algorithmic Fairness
• Consider the Alternatives: Navigating Fairness-Accuracy Tradeoffs via Disqualification