What's in a Name? Reducing Bias in Bios without Access to Protected Attributes

Abstract

There is a growing body of work that proposes methods for mitigating bias in machine learning systems. These methods typically rely on access to protected attributes such as race, gender, or age. However, this raises two significant challenges: (1) protected attributes may not be available or it may not be legal to use them, and (2) it is often desirable to simultaneously consider multiple protected attributes, as well as their intersections. In the context of mitigating bias in occupation classification, we propose a method for discouraging correlation between the predicted probability of an individual's true occupation and a word embedding of their name. This method leverages the societal biases that are encoded in word embeddings, eliminating the need for access to protected attributes. Crucially, it only requires access to individuals' names at training time and not at deployment time. We evaluate two variations of our proposed method using a large-scale dataset of online biographies. We find that both variations simultaneously reduce race and gender biases, with almost no reduction in the classifier's overall true positive rate.

arXiv commentAccepted at NAACL 2019; Best Thematic Paper

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Semantics derived automatically from language corpora contain human-like biases
• Equality of Opportunity in Supervised Learning
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Enriching Word Vectors with Subword Information
• Fairer and more accurate, but for whom?

Cited by in corpus (10)

• Measuring and Reducing Gendered Correlations in Pre-trained Models
• Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
• Two Simple Ways to Learn Individual Fairness Metrics from Data
• "What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairness
• SenSeI: Sensitive Set Invariance for Enforcing Individual Fairness
• Subverting Fair Image Search with Generative Adversarial Perturbations
• Exploring Text Specific and Blackbox Fairness Algorithms in Multimodal Clinical NLP
• Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification
• Statistical inference for individual fairness
• Information-Theoretic Bias Assessment Of Learned Representations Of Pretrained Face Recognition