Homogenization Effects of Large Language Models on Human Creative Ideation
arXiv:2402.01536 · doi:10.1145/3635636.3656204
Abstract
Large language models (LLMs) are now being used in a wide variety of contexts, including as creativity support tools (CSTs) intended to help their users come up with new ideas. But do LLMs actually support user creativity? We hypothesized that the use of an LLM as a CST might make the LLM's users feel more creative, and even broaden the range of ideas suggested by each individual user, but also homogenize the ideas suggested by different users. We conducted a 36-participant comparative user study and found, in accordance with the homogenization hypothesis, that different users tended to produce less semantically distinct ideas with ChatGPT than with an alternative CST. Additionally, ChatGPT users generated a greater number of more detailed ideas, but felt less responsible for the ideas they generated. We discuss potential implications of these findings for users, designers, and developers of LLM-based CSTs.
Accepted to C&C 2024
References in corpus (7)
- CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities
- Co-Writing with Opinionated Language Models Affects Users' Views
- Designing for Responsible Trust in AI Systems: A Communication Perspective
- A Design Space for Intelligent and Interactive Writing Assistants
- Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creators
- PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions
- Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing
Cited by in corpus (23)
- A Design Space for Intelligent and Interactive Writing Assistants
- Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking
- Assessing and Understanding Creativity in Large Language Models
- Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization
- Beyond Code Generation: LLM-supported Exploration of the Program Design Space
- From Pen to Prompt: How Creative Writers Integrate AI into their Writing Practice
- Inkspire: Supporting Design Exploration with Generative AI through Analogical Sketching
- Misty: UI Prototyping Through Interactive Conceptual Blending
- Beyond Productivity: Rethinking the Impact of Creativity Support Tools
- Timing Matters: How Using LLMs at Different Timings Influences Writers' Perceptions and Ideation Outcomes in AI-Assisted Ideation
- HarmonyCut: Supporting Creative Chinese Paper-cutting Design with Form and Connotation Harmony
- IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback
- Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbols
- One Does Not Simply Meme Alone: Evaluating Co-Creativity Between LLMs and Humans in the Generation of Humor
- Echoes in AI: Quantifying lack of plot diversity in LLM outputs
- Dancing With Chains: Ideating Under Constraints With UIDEC in UI/UX Design
- ScholarMate: A Mixed-Initiative Tool for Qualitative Knowledge Work and Information Sensemaking
- No Evidence for LLMs Being Useful in Problem Reframing
- StoryEnsemble: Enabling Dynamic Exploration & Iteration in the Design Process with AI and Forward-Backward Propagation
- Diverse AI Personas Can Mitigate the Homogenization Effect in Human-AI Collaborative Ideation
- Evaluating Sakana's AI Scientist: Bold Claims, Mixed Results, and a Promising Future?
- Constella: Supporting Storywriters' Interconnected Character Creation through LLM-based Multi-Agents
- From Planning to Revision: How AI Writing Support at Different Stages Alters Ownership