Better Together? The Role of Explanations in Supporting Novices in Individual and Collective Deliberations about AI

Abstract

Deploying AI systems in public institutions can have far-reaching consequences for many people, making it a matter of public interest. Providing opportunities for stakeholders to come together, understand these systems, and debate their merits and harms is thus essential. Explainable AI often focuses on individuals, but deliberation benefits from group settings, which are underexplored. To address this gap, we present findings from an interview study with 8 focus groups and 12 individuals. Our findings provide insight into how explanations support AI novices in deliberating alone and in groups. Participants used modular explanations with four information categories to solve tasks and decide about an AI system's deployment. We found that the explanations supported groups in creating shared understanding and in finding arguments for and against the system's deployment. In comparison, individual participants engaged with explanations in more depth and performed better in the study tasks, but missed an exchange with others. Based on our findings, we provide suggestions on how explanations should be designed to work in group settings and describe their potential use in real-world contexts. With this, our contributions inform XAI research that aims to enable AI novices to understand and deliberate AI systems in the public sector.

30 pages main text, 8 figures, 4 tables. Supplementary material is included in the appendix

References in corpus (14)

• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
• The Fallacy of AI Functionality
• Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
• Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
• Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
• Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
• The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder Early-stage Deliberations Around Public Sector AI Proposals
• Contestable Camera Cars: A Speculative Design Exploration of Public AI That Is Open and Responsive to Dispute
• Is More Always Better? The Effects of Personal Characteristics and Level of Detail on the Perception of Explanations in a Recommender System
• Question-Driven Design Process for Explainable AI User Experiences
• Information That Matters: Exploring Information Needs of People Affected by Algorithmic Decisions