Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification

Abstract

Existing bias mitigation methods to reduce disparities in model outcomes across cohorts have focused on data augmentation, debiasing model embeddings, or adding fairness-based optimization objectives during training. Separately, certified word substitution robustness methods have been developed to decrease the impact of spurious features and synonym substitutions on model predictions. While their end goals are different, they both aim to encourage models to make the same prediction for certain changes in the input. In this paper, we investigate the utility of certified word substitution robustness methods to improve equality of odds and equality of opportunity on multiple text classification tasks. We observe that certified robustness methods improve fairness, and using both robustness and bias mitigation methods in training results in an improvement in both fronts

References in corpus (9)

• Equality of Opportunity in Supervised Learning
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Adversarial Examples for Evaluating Reading Comprehension Systems
• Assessing Social and Intersectional Biases in Contextualized Word Representations
• Mitigating Gender Bias in Natural Language Processing: Literature Review
• On Adversarial Bias and the Robustness of Fair Machine Learning
• What's in a Name? Reducing Bias in Bios without Access to Protected Attributes
• Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification
• Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation