Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory

Abstract

Understanding how ML models work is a prerequisite for responsibly designing, deploying, and using ML-based systems. With interpretability approaches, ML can now offer explanations for its outputs to aid human understanding. Though these approaches rely on guidelines for how humans explain things to each other, they ultimately solve for improving the artifact -- an explanation. In this paper, we propose an alternate framework for interpretability grounded in Weick's sensemaking theory, which focuses on who the explanation is intended for. Recent work has advocated for the importance of understanding stakeholders' needs -- we build on this by providing concrete properties (e.g., identity, social context, environmental cues, etc.) that shape human understanding. We use an application of sensemaking in organizations as a template for discussing design guidelines for Sensible AI, AI that factors in the nuances of human cognition when trying to explain itself.

arXiv commentTo appear at FAccT 2022

References in corpus (15)

• A Unified Approach to Interpreting Model Predictions
• Towards A Rigorous Science of Interpretable Machine Learning
• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
• Improving fairness in machine learning systems: What do industry practitioners need?
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• Expanding Explainability: Towards Social Transparency in AI systems
• Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences
• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• Contrastive Explanation: A Structural-Model Approach
• Interpretable Classification Models for Recidivism Prediction
• Jury Learning: Integrating Dissenting Voices into Machine Learning Models
• Towards Relatable Explainable AI with the Perceptual Process

Cited by in corpus (7)

• Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
• What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks
• Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers
• Leveraging Complementary AI Explanations to Mitigate Misunderstanding in XAI
• Better Together? The Role of Explanations in Supporting Novices in Individual and Collective Deliberations about AI