Robust Optimization for Fairness with Noisy Protected Groups

Abstract

Many existing fairness criteria for machine learning involve equalizing some metric across protected groups such as race or gender. However, practitioners trying to audit or enforce such group-based criteria can easily face the problem of noisy or biased protected group information. First, we study the consequences of naively relying on noisy protected group labels: we provide an upper bound on the fairness violations on the true groups G when the fairness criteria are satisfied on noisy groups $\hat{G}$. Second, we introduce two new approaches using robust optimization that, unlike the naive approach of only relying on $\hat{G}$, are guaranteed to satisfy fairness criteria on the true protected groups G while minimizing a training objective. We provide theoretical guarantees that one such approach converges to an optimal feasible solution. Using two case studies, we show empirically that the robust approaches achieve better true group fairness guarantees than the naive approach.

To appear at 34th Conference on Neural Information Processing Systems (NeurIPS 2020); first two authors contributed equally to this work

References in corpus (11)

• Equality of Opportunity in Supervised Learning
• Counterfactual Fairness
• Fairness Without Demographics in Repeated Loss Minimization
• A Convex Framework for Fair Regression
• Learning Non-Discriminatory Predictors
• Satisfying Real-world Goals with Dataset Constraints
• Scalable Learning of Non-Decomposable Objectives
• Noise-tolerant fair classification
• Fair Learning with Private Demographic Data
• A First-Order Algorithmic Framework for Wasserstein Distributionally Robust Logistic Regression
• Deontological Ethics By Monotonicity Shape Constraints

Cited by in corpus (13)

• A Survey on the Fairness of Recommender Systems
• Large-Scale Methods for Distributionally Robust Optimization
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• Fair Learning with Private Demographic Data
• A Distributionally Robust Approach to Fair Classification
• FairBalance: How to Achieve Equalized Odds With Data Pre-processing
• Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learning
• Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation
• Fair Sequential Selection Using Supervised Learning Models
• Fairness in Forecasting of Observations of Linear Dynamical Systems
• When Fair Classification Meets Noisy Protected Attributes
• Beyond Individual and Group Fairness
• Optimizing Black-box Metrics with Iterative Example Weighting