Fairness Without Demographics in Repeated Loss Minimization

Abstract

Machine learning models (e.g., speech recognizers) are usually trained to minimize average loss, which results in representation disparity---minority groups (e.g., non-native speakers) contribute less to the training objective and thus tend to suffer higher loss. Worse, as model accuracy affects user retention, a minority group can shrink over time. In this paper, we first show that the status quo of empirical risk minimization (ERM) amplifies representation disparity over time, which can even make initially fair models unfair. To mitigate this, we develop an approach based on distributionally robust optimization (DRO), which minimizes the worst case risk over all distributions close to the empirical distribution. We prove that this approach controls the risk of the minority group at each time step, in the spirit of Rawlsian distributive justice, while remaining oblivious to the identity of the groups. We demonstrate that DRO prevents disparity amplification on examples where ERM fails, and show improvements in minority group user satisfaction in a real-world text autocomplete task.

arXiv commentFinal version for ICML2018, corrects typos

Cited by in corpus (89)

• On the Opportunities and Risks of Foundation Models
• Shortcut Learning in Deep Neural Networks
• The Measure and Mismeasure of Fairness
• Gender bias and stereotypes in Large Language Models
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Fair Resource Allocation in Federated Learning
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Towards Out-Of-Distribution Generalization: A Survey
• Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
• Ditto: Fair and Robust Federated Learning Through Personalization
• Evaluating Stochastic Rankings with Expected Exposure
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Training individually fair ML models with Sensitive Subspace Robustness
• Fairness without Demographics through Adversarially Reweighted Learning
• Mitigating Gender Bias in Natural Language Processing: Literature Review
• Fairness for Robust Log Loss Classification
• Robust Optimization for Fairness with Noisy Protected Groups
• How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
• RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network
• Towards Fair Classifiers Without Sensitive Attributes: Exploring Biases in Related Features
• Large-Scale Methods for Distributionally Robust Optimization
• FairBatch: Batch Selection for Model Fairness
• A Survey on Intersectional Fairness in Machine Learning: Notions, Mitigation, and Challenges
• Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
• Noise-tolerant fair classification
• Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• Algorithmic Decision Making with Conditional Fairness
• Fair Learning with Private Demographic Data
• Fairness without the sensitive attribute via Causal Variational Autoencoder
• Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity
• Training Fair Models in Federated Learning without Data Privacy Infringement
• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
• Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction
• Debiased Contrastive Learning
• Equalized odds postprocessing under imperfect group information
• Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning
• Fair Generative Modeling via Weak Supervision
• Performative Prediction
• How Do Fair Decisions Fare in Long-term Qualification?
• Gradient Matching for Domain Generalization
• Measuring justice in machine learning
• Utility is in the Eye of the User: A Critique of NLP Leaderboards
• FR-Train: A Mutual Information-Based Approach to Fair and Robust Training
• Bias and Fairness in Computer Vision Applications of the Criminal Justice System
• Causal Modeling for Fairness in Dynamical Systems
• Environment Inference for Invariant Learning
• On Tilted Losses in Machine Learning: Theory and Applications
• Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective
• Modeling Techniques for Machine Learning Fairness: A Survey
• Fair Classification with Noisy Protected Attributes: A Framework with Provable Guarantees
• Fairness in Forecasting and Learning Linear Dynamical Systems
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Coping with Label Shift via Distributionally Robust Optimisation
• Fairness-aware Agnostic Federated Learning
• Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data
• Distributionally Robust Bayesian Quadrature Optimization
• Group fairness without demographics using social networks
• Selective Classification Can Magnify Disparities Across Groups
• Adversarial Reweighting for Speaker Verification Fairness
• Understanding and Mitigating Accuracy Disparity in Regression
• Tilted Empirical Risk Minimization
• Towards Principled Disentanglement for Domain Generalization
• Learning with risks based on M-location
• When Fair Classification Meets Noisy Protected Attributes
• Contextual Prediction Difference Analysis for Explaining Individual Image Classifications
• Focus on the Common Good: Group Distributional Robustness Follows
• Fairness Through Causal Awareness: Learning Latent-Variable Models for Biased Data
• FedPrune: Towards Inclusive Federated Learning
• A Survey of Learning Criteria Going Beyond the Usual Risk
• Towards Return Parity in Markov Decision Processes
• Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach
• Statistical inference for individual fairness
• Exploring the Unfairness of DP-SGD Across Settings
• Fair Classification with Adversarial Perturbations
• A Tale Of Two Long Tails
• A Distributed Fair Machine Learning Framework with Private Demographic Data Protection
• From Fair Decision Making to Social Equality
• Understanding Distributional Ambiguity via Non-robust Chance Constraint
• Contrastive Clustering: Toward Unsupervised Bias Reduction for Emotion and Sentiment Classification
• On the Fairness of Randomized Trials for Recommendation with Heterogeneous Demographics and Beyond
• Individually Fair Gradient Boosting
• Nonlinear Monte Carlo Method for Imbalanced Data Learning
• Combating Unknown Bias with Effective Bias-Conflicting Scoring and Gradient Alignment
• Model-agnostic bias mitigation methods with regressor distribution control for Wasserstein-based fairness metrics
• Distributionally Robust Graphical Models
• Boosted CVaR Classification
• Deep Clustering based Fair Outlier Detection
• Distributionally Robust Language Modeling