Cultural Bias and Cultural Alignment of Large Language Models

Abstract

Culture fundamentally shapes people's reasoning, behavior, and communication. As people increasingly use generative artificial intelligence (AI) to expedite and automate personal and professional tasks, cultural values embedded in AI models may bias people's authentic expression and contribute to the dominance of certain cultures. We conduct a disaggregated evaluation of cultural bias for five widely used large language models (OpenAI's GPT-4o/4-turbo/4/3.5-turbo/3) by comparing the models' responses to nationally representative survey data. All models exhibit cultural values resembling English-speaking and Protestant European countries. We test cultural prompting as a control strategy to increase cultural alignment for each country/territory. For recent models (GPT-4, 4-turbo, 4o), this improves the cultural alignment of the models' output for 71-81% of countries and territories. We suggest using cultural prompting and ongoing evaluation to reduce cultural bias in the output of generative AI.

References in corpus (1)

• Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models

Cited by in corpus (15)

• AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
• ExploreSelf: Fostering User-driven Exploration and Reflection on Personal Challenges with Adaptive Guidance by Large Language Models
• A Survey on Moral Foundation Theory and Pre-Trained Language Models: Current Advances and Challenges
• Artificial Intelligence Can Emulate Human Normative Judgments on Emotional Visual Scenes
• Six Fallacies in Substituting Large Language Models for Human Participants
• AI-generated stories favour stability over change: homogeneity and cultural stereotyping in narratives generated by gpt-4o-mini
• Ontologies in Design: How Imagining a Tree Reveals Possibilites and Assumptions in Large Language Models
• Did Alice Do Wrong? Cross-Cultural Differences in Student Perceptions of Generative AI Use in University Computing Education
• Computational Hermeneutics: Evaluating generative AI as a cultural technology
• Large Language Models as Psychological Simulators: A Methodological Guide
• The Fair Game: Auditing & Debiasing AI Algorithms Over Time
• Beyond Partisan Leaning: A Comparative Analysis of Political Bias in Large Language Models
• AI-generated podcasts: Synthetic Intimacy and Cultural Mistranslation in NotebookLM's Audio Overviews
• "Draw me a curator" Examining the visual stereotyping of a cultural services profession by generative AI
• Towards culturally-appropriate conversational AI for health in the majority world: An exploratory study with citizens and professionals in Latin America