Ensuring Fairness Beyond the Training Data
arXiv:2007.06029
Abstract
We initiate the study of fair classifiers that are robust to perturbations in the training distribution. Despite recent progress, the literature on fairness has largely ignored the design of fair and robust classifiers. In this work, we develop classifiers that are fair not only with respect to the training distribution, but also for a class of distributions that are weighted perturbations of the training samples. We formulate a min-max objective function whose goal is to minimize a distributionally robust training loss, and at the same time, find a classifier that is fair with respect to a class of distributions. We first reduce this problem to finding a fair classifier that is robust with respect to the class of distributions. Based on online learning algorithm, we develop an iterative algorithm that provably converges to such a fair and robust solution. Experiments on standard machine learning fairness datasets suggest that, compared to the state-of-the-art fair classifiers, our classifier retains fairness guarantees and test accuracy for a large class of perturbations on the test set. Furthermore, our experiments show that there is an inherent trade-off between fairness robustness and accuracy of such classifiers.
arXiv comment18 pages, 3 figures, To appear at NeurIPS-2020
References in corpus (10)
- Scikit-learn: Machine Learning in Python
- Equality of Opportunity in Supervised Learning
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
- Counterfactual Fairness
- Fairness Testing: Testing Software for Discrimination
- Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
- On Fairness and Calibration
- AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
- Distributionally Robust Logistic Regression
- Unleashing Linear Optimizers for Group-Fair Learning and Optimization