Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI

Abstract

Explainable AI (XAI) systems are sociotechnical in nature; thus, they are subject to the sociotechnical gap--divide between the technical affordances and the social needs. However, charting this gap is challenging. In the context of XAI, we argue that charting the gap improves our problem understanding, which can reflexively provide actionable insights to improve explainability. Utilizing two case studies in distinct domains, we empirically derive a framework that facilitates systematic charting of the sociotechnical gap by connecting AI guidelines in the context of XAI and elucidating how to use them to address the gap. We apply the framework to a third case in a new domain, showcasing its affordances. Finally, we discuss conceptual implications of the framework, share practical considerations in its operationalization, and offer guidance on transferring it to new contexts. By making conceptual and practical contributions to understanding the sociotechnical gap in XAI, the framework expands the XAI design space.

arXiv commentPublished at ACM CSCW 2023

References in corpus (22)

• Towards A Rigorous Science of Interpretable Machine Learning
• Model Cards for Model Reporting
• Understanding Neural Networks Through Deep Visualization
• Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
• Explaining Explanations in AI
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• What do we need to build explainable AI systems for the medical domain?
• Expanding Explainability: Towards Social Transparency in AI systems
• One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
• Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
• Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches
• Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes
• CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysis
• Machine Learning Model Interpretability for Precision Medicine
• In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes
• Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions
• Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study
• Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems
• Question-Driven Design Process for Explainable AI User Experiences
• Nine Potential Pitfalls when Designing Human-AI Co-Creative Systems
• The Sanction of Authority: Promoting Public Trust in AI

Cited by in corpus (13)

• Artificial Intelligence Should Genuinely Support Clinical Reasoning and Decision Making To Bridge the Translational Gap
• Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• Are We Asking the Right Questions?: Designing for Community Stakeholders' Interactions with AI in Policing
• What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks
• Perceptions of Sentient AI and Other Digital Minds: Evidence from the AI, Morality, and Sentience (AIMS) Survey
• Explainable AI Reloaded: Challenging the XAI Status Quo in the Era of Large Language Models
• Information That Matters: Exploring Information Needs of People Affected by Algorithmic Decisions
• Balancing Caregiving and Self-Care: Exploring Mental Health Needs of Alzheimer's and Dementia Caregivers
• Give Me a Choice: The Consequences of Restricting Choices Through AI-Support for Perceived Autonomy, Motivational Variables, and Decision Performance
• Design Considerations for Human Oversight of AI: Insights from Co-Design Workshops and Work Design Theory
• Better Together? The Role of Explanations in Supporting Novices in Individual and Collective Deliberations about AI
• Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Study