Visual Story-Writing: Writing by Manipulating Visual Representations of Stories

Abstract

We define "visual story-writing" as using visual representations of story elements to support writing and revising narrative texts. To demonstrate this approach, we developed a text editor that automatically visualizes a graph of entity interactions, movement between locations, and a timeline of story events. Interacting with these visualizations results in suggested text edits: for example, connecting two characters in the graph creates an interaction between them, moving an entity updates their described location, and rearranging events on the timeline reorganizes the narrative sequence. Through two user studies on narrative text editing and writing, we found that visuals supported participants in planning high-level revisions, tracking story elements, and exploring story variations in ways that encourage creativity. Broadly, our work lays the foundation for writing support, not just through words, but also visuals.

In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST '25)

References in corpus (19)

• The emotional arcs of stories are dominated by six basic shapes
• Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation
• A Design Space for Intelligent and Interactive Writing Assistants
• The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
• DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models
• VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping
• Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
• Sketch-n-Sketch: Output-Directed Programming for SVG
• Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting
• ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Models
• Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing
• Bidirectional Evaluation with Direct Manipulation
• Portrayal: Leveraging NLP and Visualization for Analyzing Fictional Characters
• Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools
• Textoshop: Interactions Inspired by Drawing Software to Facilitate Text Editing
• Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbols
• Making the Write Connections: Linking Writing Support Tools with Writer's Needs
• FairyTailor: A Multimodal Generative Framework for Storytelling
• Agents' Room: Narrative Generation through Multi-step Collaboration