Exploring Mobile Touch Interaction with Large Language Models
arXiv:2502.07629 · doi:10.1145/3706598.3713554
Abstract
Interacting with Large Language Models (LLMs) for text editing on mobile devices currently requires users to break out of their writing environment and switch to a conversational AI interface. In this paper, we propose to control the LLM via touch gestures performed directly on the text. We first chart a design space that covers fundamental touch input and text transformations. In this space, we then concretely explore two control mappings: spread-to-generate and pinch-to-shorten, with visual feedback loops. We evaluate this concept in a user study (N=14) that compares three feedback designs: no visualisation, text length indicator, and length + word indicator. The results demonstrate that touch-based control of LLMs is both feasible and user-friendly, with the length + word indicator proving most effective for managing text generation. This work lays the foundation for further research into gesture-based interaction with LLMs on touch devices.
21 pages, 16 figures, 3 tables, ACM CHI 2025
References in corpus (10)
- The AI Ghostwriter Effect: When Users Do Not Perceive Ownership of AI-Generated Text But Self-Declare as Authors
- A Design Space for Intelligent and Interactive Writing Assistants
- DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models
- Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting
- From Text to Self: Users' Perceptions of Potential of AI on Interpersonal Communication and Self
- Writer-Defined AI Personas for On-Demand Feedback Generation
- Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing
- Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization
- Suggestion Lists vs. Continuous Generation: Interaction Design for Writing with Generative Models on Mobile Devices Affect Text Length, Wording and Perceived Authorship
- Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools