Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humans

Abstract

Large language models (LLMs) are becoming pervasive in everyday life, yet their propensity to reproduce biases inherited from training data remains a pressing concern. Prior investigations into bias in LLMs have focused on the association of social groups with stereotypical attributes. However, this is only one form of human bias such systems may reproduce. We investigate a new form of bias in LLMs that resembles a social psychological phenomenon where socially subordinate groups are perceived as more homogeneous than socially dominant groups. We had ChatGPT, a state-of-the-art LLM, generate texts about intersectional group identities and compared those texts on measures of homogeneity. We consistently found that ChatGPT portrayed African, Asian, and Hispanic Americans as more homogeneous than White Americans, indicating that the model described racial minority groups with a narrower range of human experience. ChatGPT also portrayed women as more homogeneous than men, but these differences were small. Finally, we found that the effect of gender differed across racial/ethnic groups such that the effect of gender was consistent within African and Hispanic Americans but not within Asian and White Americans. We argue that the tendency of LLMs to describe groups as less diverse risks perpetuating stereotypes and discriminatory behavior.

Forthcoming at ACM Conference on Fairness, Accountability, and Transparency (FAccT) 2024

References in corpus (7)

• Semantics derived automatically from language corpora contain human-like biases
• Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
• What you can cram into a single vector: Probing sentence embeddings for linguistic properties
• Runaway Feedback Loops in Predictive Policing
• Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
• Persistent Anti-Muslim Bias in Large Language Models
• On Measures of Biases and Harms in NLP

Cited by in corpus (7)

• Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)
• Six Fallacies in Substituting Large Language Models for Human Participants
• Not Like Us, Hunty: Measuring Perceptions and Behavioral Effects of Minoritized Anthropomorphic Cues in LLMs
• Laissez-Faire Harms: Algorithmic Biases in Generative Language Models
• Can Large Language Models Grasp Concepts in Visual Content? A Case Study on YouTube Shorts about Depression
• Large Language Models as Psychological Simulators: A Methodological Guide
• Valuing Time in Silicon: Can Large Language Models Replicate Human Value of Travel Time