Assessing Social and Intersectional Biases in Contextualized Word Representations

Abstract

Social bias in machine learning has drawn significant attention, with work ranging from demonstrations of bias in a multitude of applications, curating definitions of fairness for different contexts, to developing algorithms to mitigate bias. In natural language processing, gender bias has been shown to exist in context-free word embeddings. Recently, contextual word representations have outperformed word embeddings in several downstream NLP tasks. These word representations are conditioned on their context within a sentence, and can also be used to encode the entire sentence. In this paper, we analyze the extent to which state-of-the-art models for contextual word representations, such as BERT and GPT-2, encode biases with respect to gender, race, and intersectional identities. Towards this, we propose assessing bias at the contextual word level. This novel approach captures the contextual effects of bias missing in context-free word embeddings, yet avoids confounding effects that underestimate bias at the sentence encoding level. We demonstrate evidence of bias at the corpus level, find varying evidence of bias in embedding association tests, show in particular that racial bias is strongly encoded in contextual word models, and observe that bias effects for intersectional minorities are exacerbated beyond their constituent minority identities. Further, evaluating bias effects at the contextual word level captures biases that are not captured at the sentence level, confirming the need for our novel approach.

NeurIPS 2019

Cited by in corpus (13)

• Large image datasets: A pyrrhic win for computer vision?
• Alignment of Language Agents
• Taking a Stance on Fake News: Towards Automatic Disinformation Assessment via Deep Bidirectional Transformer Language Models for Stance Detection
• Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models
• Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models
• Debiasing Pre-trained Contextualised Embeddings
• Disembodied Machine Learning: On the Illusion of Objectivity in NLP
• Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification
• Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies
• Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification
• The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability
• Low-Complexity Probing via Finding Subnetworks
• Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?