Undesirable Biases in NLP: Addressing Challenges of Measurement

Abstract

As Large Language Models and Natural Language Processing (NLP) technology rapidly develop and spread into daily life, it becomes crucial to anticipate how their use could harm people. One problem that has received a lot of attention in recent years is that this technology has displayed harmful biases, from generating derogatory stereotypes to producing disparate outcomes for different social groups. Although a lot of effort has been invested in assessing and mitigating these biases, our methods of measuring the biases of NLP models have serious problems and it is often unclear what they actually measure. In this paper, we provide an interdisciplinary approach to discussing the issue of NLP model bias by adopting the lens of psychometrics -- a field specialized in the measurement of concepts like bias that are not directly observable. In particular, we will explore two central notions from psychometrics, the construct validity and the reliability of measurement tools, and discuss how they can be applied in the context of measuring model bias. Our goal is to provide NLP practitioners with methodological tools for designing better bias measures, and to inspire them more generally to explore tools from psychometrics when working on bias measurement tools.

References in corpus (29)

• On the Opportunities and Risks of Foundation Models
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Multitask Prompted Training Enables Zero-Shot Task Generalization
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
• Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches
• The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
• Measuring and Reducing Gendered Correlations in Pre-trained Models
• A Survey on Gender Bias in Natural Language Processing
• Socially Responsible AI Algorithms: Issues, Purposes, and Challenges
• How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence
• Time Travel in LLMs: Tracing Data Contamination in Large Language Models
• PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
• Robustness and Reliability of Gender Bias Assessment in Word Embeddings: The Role of Base Pairs
• Social Biases in NLP Models as Barriers for Persons with Disabilities
• Evaluating the Construct Validity of Text Embeddings with Application to Survey Questions
• On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations
• Quantifying Social Biases Using Templates is Unreliable
• Trustworthy Social Bias Measurement
• Cross-replication Reliability -- An Empirical Approach to Interpreting Inter-rater Reliability
• Challenges in Measuring Bias via Open-Ended Language Generation
• On Measuring Social Biases in Prompt-Based Multi-Task Learning
• Sparse Interventions in Language Models with Differentiable Masking
• The Birth of Bias: A case study on the evolution of gender bias in an English language model
• Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information
• This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Models
• Analyzing Gender Representation in Multilingual Models
• Gender Bias Hidden Behind Chinese Word Embeddings: The Case of Chinese Adjectives