UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital Library

Abstract

Early conversational agents (CAs) focused on dyadic human-AI interaction between humans and the CAs, followed by the increasing popularity of polyadic human-AI interaction, in which CAs are designed to mediate human-human interactions. CAs for polyadic interactions are unique because they encompass hybrid social interactions, i.e., human-CA, human-to-human, and human-to-group behaviors. However, research on polyadic CAs is scattered across different fields, making it challenging to identify, compare, and accumulate existing knowledge. To promote the future design of CA systems, we conducted a literature review of ACM publications and identified a set of works that conducted UX (user experience) research. We qualitatively synthesized the effects of polyadic CAs into four aspects of human-human interactions, i.e., communication, engagement, connection, and relationship maintenance. Through a mixed-method analysis of the selected polyadic and dyadic CA studies, we developed a suite of evaluation measurements on the effects. Our findings show that designing with social boundaries, such as privacy, disclosure, and identification, is crucial for ethical polyadic CAs. Future research should also advance usability testing methods and trust-building guidelines for conversational AI.

References in corpus (6)

• Conceptual Metaphors Impact Perceptions of Human-AI Collaboration
• If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills
• Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop
• Effects of Persuasive Dialogues: Testing Bot Identities and Inquiry Strategies
• Emotionally-Aware Chatbots: A Survey
• Pipelines for Procedural Information Extraction from Scientific Literature: Towards Recipes using Machine Learning and Data Science