Reducing Sentiment Bias in Language Models via Counterfactual Evaluation

Abstract

Advances in language modeling architectures and the availability of large text corpora have driven progress in automatic text generation. While this results in models capable of generating coherent texts, it also prompts models to internalize social biases present in the training corpus. This paper aims to quantify and reduce a particular type of bias exhibited by language models: bias in the sentiment of generated text. Given a conditioning context (e.g., a writing prompt) and a language model, we analyze if (and how) the sentiment of the generated text is affected by changes in values of sensitive attributes (e.g., country names, occupations, genders) in the conditioning context using a form of counterfactual evaluation. We quantify sentiment bias by adopting individual and group fairness metrics from the fair machine learning literature, and demonstrate that large-scale models trained on two different corpora (news articles, and Wikipedia) exhibit considerable levels of bias. We then propose embedding and sentiment prediction-derived regularization on the language model's latent representations. The regularizations improve fairness metrics while retaining comparable levels of perplexity and semantic similarity.

arXiv commentAccepted in the Findings of EMNLP, 2020

References in corpus (19)

• Efficient Estimation of Word Representations in Vector Space
• Semantics derived automatically from language corpora contain human-like biases
• Equality of Opportunity in Supervised Learning
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• Universal Sentence Encoder
• Counterfactual Fairness
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• Pointer Sentinel Mixture Models
• Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
• Release Strategies and the Social Impacts of Language Models
• Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
• Multi-Task Deep Neural Networks for Natural Language Understanding
• Flexibly Fair Representation Learning by Disentanglement
• Women also Snowboard: Overcoming Bias in Captioning Models
• DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
• Learning Gender-Neutral Word Embeddings
• Wasserstein Fair Classification
• Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation
• Certified Robustness to Adversarial Word Substitutions

Cited by in corpus (7)

• Language Models are Few-Shot Learners
• Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
• Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Tasks
• Alignment of Language Agents
• Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond
• Investigating Societal Biases in a Poetry Composition System
• A Sparsity Algorithm with Applications to Corporate Credit Rating