XAIR: A Framework of Explainable AI in Augmented Reality

Abstract

Explainable AI (XAI) has established itself as an important component of AI-driven interactive systems. With Augmented Reality (AR) becoming more integrated in daily lives, the role of XAI also becomes essential in AR because end-users will frequently interact with intelligent services. However, it is unclear how to design effective XAI experiences for AR. We propose XAIR, a design framework that addresses "when", "what", and "how" to provide explanations of AI output in AR. The framework was based on a multi-disciplinary literature review of XAI and HCI research, a large-scale survey probing 500+ end-users' preferences for AR-based explanations, and three workshops with 12 experts collecting their insights about XAI design in AR. XAIR's utility and effectiveness was verified via a study with 10 designers and another study with 12 end-users. XAIR can provide guidelines for designers, inspiring them to identify new design opportunities and achieve effective XAI designs in AR.

arXiv commentProceedings of the 2023 CHI Conference on Human Factors in Computing Systems

References in corpus (24)

• Learning Transferable Visual Models From Natural Language Supervision
• Towards A Rigorous Science of Interpretable Machine Learning
• Learning Important Features Through Propagating Activation Differences
• Understanding Black-box Predictions via Influence Functions
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Explaining Explanations in AI
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• Explainable Recommendation: A Survey and New Perspectives
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
• What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
• Towards Explainable Artificial Intelligence
• Expanding Explainability: Towards Social Transparency in AI systems
• Human-in-the-loop Artificial Intelligence
• One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
• Deep Learning for Generic Object Detection: A Survey
• The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
• CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysis
• Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery
• How Case Based Reasoning Explained Neural Networks: An XAI Survey of Post-Hoc Explanation-by-Example in ANN-CBR Twins
• Impacts of Personal Characteristics on User Trust in Conversational Recommender Systems
• Intent-Aware Contextual Recommendation System

Cited by in corpus (11)

• GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Reality
• Augmented Object Intelligence with XR-Objects
• Perceptions of Sentient AI and Other Digital Minds: Evidence from the AI, Morality, and Sentience (AIMS) Survey
• Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents
• Wearable AR in Everyday Contexts: Insights from a Digital Ethnography of YouTube Videos
• TOM: A Development Platform For Wearable Intelligent Assistants
• Designing for Human Operations on the Moon: Challenges and Opportunities of Navigational HUD Interfaces
• PILAR: Personalizing Augmented Reality Interactions with LLM-based Human-Centric and Trustworthy Explanations for Daily Use Cases
• Co-Designing Digital Humans for Online Learning: A Framework for Human-AI Pedagogical Integration
• SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Reality
• Demonstrating PilotAR: A Tool to Assist Wizard-of-Oz Pilot Studies with OHMD