Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation
arXiv:2310.12953 · doi:10.1145/3613904.3642400
Abstract
Thanks to their generative capabilities, large language models (LLMs) have become an invaluable tool for creative processes. These models have the capacity to produce hundreds and thousands of visual and textual outputs, offering abundant inspiration for creative endeavors. But are we harnessing their full potential? We argue that current interaction paradigms fall short, guiding users towards rapid convergence on a limited set of ideas, rather than empowering them to explore the vast latent design space in generative models. To address this limitation, we propose a framework that facilitates the structured generation of design space in which users can seamlessly explore, evaluate, and synthesize a multitude of responses. We demonstrate the feasibility and usefulness of this framework through the design and development of an interactive system, Luminate, and a user study with 14 professional writers. Our work advances how we interact with LLMs for creative tasks, introducing a way to harness the creative potential of LLMs.
References in corpus (8)
- Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models
- Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools
- Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
- CodeToon: Story Ideation, Auto Comic Generation, and Structure Mapping for Code-Driven Storytelling
- CrossCode: Multi-level Visualization of Program Execution
- Mapping the Design Space of Interactions in Human-AI Text Co-creation Tasks
- ReelFramer: Human-AI Co-Creation for News-to-Video Translation
- Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models
Cited by in corpus (35)
- Augmented Object Intelligence with XR-Objects
- Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking
- Malleable Overview-Detail Interfaces
- AI-Instruments: Embodying Prompts as Instruments to Abstract & Reflect Graphical Interface Commands as General-Purpose Tools
- Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
- Are We On Track? AI-Assisted Active and Passive Goal Reflection During Meetings
- DesignWeaver: Dimensional Scaffolding for Text-to-Image Product Design
- Beyond Productivity: Rethinking the Impact of Creativity Support Tools
- HarmonyCut: Supporting Creative Chinese Paper-cutting Design with Form and Connotation Harmony
- DesignFromX: Empowering Consumer-Driven Design Space Exploration through Feature Composition of Referenced Products
- Textoshop: Interactions Inspired by Drawing Software to Facilitate Text Editing
- VideoDiff: Human-AI Video Co-Creation with Alternatives
- Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulation
- Expandora: Broadening Design Exploration with Text-to-Image Model
- PromptMap: An Alternative Interaction Style for AI-Based Image Generation
- SPROUT: an Interactive Authoring Tool for Generating Programming Tutorials with the Visualization of Large Language Models
- NeuroSync: Intent-Aware Code-Based Problem Solving via Direct LLM Understanding Modification
- Intent Tagging: Exploring Micro-Prompting Interactions for Supporting Granular Human-GenAI Co-Creation Workflows
- Can GPT-4o Evaluate Usability Like Human Experts? A Comparative Study on Issue Identification in Heuristic Evaluation
- Composable Prompting Workspaces for Creative Writing: Exploration and Iteration Using Dynamic Widgets
- No Evidence for LLMs Being Useful in Problem Reframing
- GenPara: Enhancing the 3D Design Editing Process by Inferring Users' Regions of Interest with Text-Conditional Shape Parameters
- StoryEnsemble: Enabling Dynamic Exploration & Iteration in the Design Process with AI and Forward-Backward Propagation
- Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions
- ThematicPlane: Bridging Tacit User Intent and Latent Spaces for Image Generation
- PaperBridge: Crafting Research Narratives through Human-AI Co-Exploration
- Vipera: Towards systematic auditing of generative text-to-image models at scale
- KinemaFX: A Kinematic-Driven Interactive System for Particle Effect Exploration and Customization
- Varif.ai to Vary and Verify User-Driven Diversity in Scalable Image Generation
- Visual Story-Writing: Writing by Manipulating Visual Representations of Stories
- Mapping and Comparing Climate Equity Policy Practices Using RAG LLM-Based Semantic Analysis and Recommendation Systems
- StudyAlign: A Software System for Conducting Web-Based User Studies with Functional Interactive Prototypes
- Spatial Balancing: Designing an LLM-Powered Spatial Externalization Interface for Iterative Science Communication Writing
- From Planning to Revision: How AI Writing Support at Different Stages Alters Ownership
- OOPrompt: Reifying Intents into Structured Artifacts for Modular and Iterative Prompting