Should AI Mimic People? Understanding AI-Supported Writing Technology Among Black Users

Abstract

AI-supported writing technologies (AISWT) that provide grammatical suggestions, autocomplete sentences, or generate and rewrite text are now a regular feature integrated into many people's workflows. However, little is known about how people perceive the suggestions these tools provide. In this paper, we investigate how Black American users perceive AISWT, motivated by prior findings in natural language processing that highlight how the underlying large language models can contain racial biases. Using interviews and observational user studies with 13 Black American users of AISWT, we found a strong tradeoff between the perceived benefits of using AISWT to enhance their writing style and feeling like "it wasn't built for us". Specifically, participants reported AISWT's failure to recognize commonly used names and expressions in African American Vernacular English, experiencing its corrections as hurtful and alienating and fearing it might further minoritize their culture. We end with a reflection on the tension between AISWT that fail to include Black American culture and language, and AISWT that attempt to mimic it, with attention to accuracy, authenticity, and the production of social difference.

accepted to CSCW 2025

References in corpus (14)

• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
• Human heuristics for AI-generated language are flawed
• Co-Writing with Opinionated Language Models Affects Users' Views
• Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development
• "It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models
• AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
• The Capacity for Moral Self-Correction in Large Language Models
• Cultural Incongruencies in Artificial Intelligence
• Blaming Humans and Machines: What Shapes People's Reactions to Algorithmic Harm
• Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation
• Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework
• What drives the acceptance of AI technology?: the role of expectations and experiences
• How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensions