Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory

Abstract

Understanding how ML models work is a prerequisite for responsibly designing, deploying, and using ML-based systems. With interpretability approaches, ML can now offer explanations for its outputs to aid human understanding. Though these approaches rely on guidelines for how humans explain things to each other, they ultimately solve for improving the artifact -- an explanation. In this paper, we propose an alternate framework for interpretability grounded in Weick's sensemaking theory, which focuses on who the explanation is intended for. Recent work has advocated for the importance of understanding stakeholders' needs -- we build on this by providing concrete properties (e.g., identity, social context, environmental cues, etc.) that shape human understanding. We use an application of sensemaking in organizations as a template for discussing design guidelines for Sensible AI, AI that factors in the nuances of human cognition when trying to explain itself.

To appear at FAccT 2022

References in corpus (8)

• Towards A Rigorous Science of Interpretable Machine Learning
• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
• Improving fairness in machine learning systems: What do industry practitioners need?
• Expanding Explainability: Towards Social Transparency in AI systems
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences
• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• Jury Learning: Integrating Dissenting Voices into Machine Learning Models