User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistants
arXiv:2309.13879 · doi:10.1016/j.ijhcs.2024.103406
Abstract
Conventional Voice Assistants (VAs) rely on traditional language models to discern user intent and respond to their queries, leading to interactions that often lack a broader contextual understanding, an area in which Large Language Models (LLMs) excel. However, current LLMs are largely designed for text-based interactions, thus making it unclear how user interactions will evolve if their modality is changed to voice. In this work, we investigate whether LLMs can enrich VA interactions via an exploratory study with participants (N=20) using a ChatGPT-powered VA for three scenarios (medical self-diagnosis, creative planning, and discussion) with varied constraints, stakes, and objectivity. We observe that LLM-powered VA elicits richer interaction patterns that vary across tasks, showing its versatility. Notably, LLMs absorb the majority of VA intent recognition failures. We additionally discuss the potential of harnessing LLMs for more resilient and fluid user-VA interactions and provide design guidelines for tailoring LLMs for voice assistance.
References in corpus (11)
- Showing Academic Performance Predictions during Term Planning: Effects on Students' Decisions, Behaviors, and Preferences
- Co-Writing with Opinionated Language Models Affects Users' Views
- Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions
- Let's have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations
- Mapping Perceptions of Humanness in Speech-Based Intelligent Personal Assistant Interaction
- Eliciting and Analysing Users' Envisioned Dialogues with Perfect Voice Assistants
- Learning gain differences between ChatGPT and human tutor generated algebra hints
- MIDAS: A Dialog Act Annotation Scheme for Open Domain Human Machine Spoken Conversations
- Gender Biases in Error Mitigation by Voice Assistants
- The Partner Modelling Questionnaire: A validated self-report measure of perceptions toward machines as dialogue partners
- Towards Building Voice-based Conversational Recommender Systems: Datasets, Potential Solutions, and Prospects