Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulation
arXiv:2401.10838 · doi:10.1145/3613904.3642217
Abstract
Dictation enables efficient text input on mobile devices. However, writing with speech can produce disfluent, wordy, and incoherent text and thus requires heavy post-processing. This paper presents Rambler, an LLM-powered graphical user interface that supports gist-level manipulation of dictated text with two main sets of functions: gist extraction and macro revision. Gist extraction generates keywords and summaries as anchors to support the review and interaction with spoken text. LLM-assisted macro revisions allow users to respeak, split, merge and transform dictated text without specifying precise editing locations. Together they pave the way for interactive dictation and revision that help close gaps between spontaneous spoken words and well-structured writing. In a comparative study with 12 participants performing verbal composition tasks, Rambler outperformed the baseline of a speech-to-text editor + ChatGPT, as it better facilitates iterative revisions with enhanced user control over the content while supporting surprisingly diverse user strategies.
arXiv commentTo appear at ACM CHI 2024
References in corpus (5)
- Comparing Speech and Keyboard Text Entry for Short Messages in Two Languages on Touchscreen Phones
- VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping
- Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
- Improving Readability for Automatic Speech Recognition Transcription
- Typist Experiment: an Investigation of Human-to-Human Dictation via Role-play to Inform Voice-based Text Authoring
Cited by in corpus (12)
- Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
- Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools
- Bridging Generations using AI-Supported Co-Creative Activities
- Textoshop: Interactions Inspired by Drawing Software to Facilitate Text Editing
- Making the Write Connections: Linking Writing Support Tools with Writer's Needs
- No Evidence for LLMs Being Useful in Problem Reframing
- Content-Driven Local Response: Supporting Sentence-Level and Message-Level Mobile Email Replies With and Without AI
- StepWrite: Adaptive Planning for Speech-Driven Text Generation
- From Voice to Value: Leveraging AI to Enhance Spoken Online Reviews on the Go
- TalkLess: Blending Extractive and Abstractive Speech Summarization for Editing Speech to Preserve Content and Style
- Script2Screen: Supporting Dialogue Scriptwriting with Interactive Audiovisual Generation
- NoTeeline: Supporting Real-Time, Personalized Notetaking with LLM-Enhanced Micronotes