Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI

Abstract

Explainable AI (XAI) systems are sociotechnical in nature; thus, they are subject to the sociotechnical gap--divide between the technical affordances and the social needs. However, charting this gap is challenging. In the context of XAI, we argue that charting the gap improves our problem understanding, which can reflexively provide actionable insights to improve explainability. Utilizing two case studies in distinct domains, we empirically derive a framework that facilitates systematic charting of the sociotechnical gap by connecting AI guidelines in the context of XAI and elucidating how to use them to address the gap. We apply the framework to a third case in a new domain, showcasing its affordances. Finally, we discuss conceptual implications of the framework, share practical considerations in its operationalization, and offer guidance on transferring it to new contexts. By making conceptual and practical contributions to understanding the sociotechnical gap in XAI, the framework expands the XAI design space.

Published at ACM CSCW 2023

References in corpus (17)

• Towards A Rigorous Science of Interpretable Machine Learning
• Understanding Neural Networks Through Deep Visualization
• Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• What do we need to build explainable AI systems for the medical domain?
• Expanding Explainability: Towards Social Transparency in AI systems
• Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment
• Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches
• Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes
• CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysis
• Machine Learning Model Interpretability for Precision Medicine
• In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes
• Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions
• Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study
• Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems
• Nine Potential Pitfalls when Designing Human-AI Co-Creative Systems
• The Sanction of Authority: Promoting Public Trust in AI

Cited by in corpus (3)

• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• Are We Asking the Right Questions?: Designing for Community Stakeholders' Interactions with AI in Policing
• What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks