Towards an Explanation Space to Align Humans and Explainable-AI Teamwork
arXiv:2106.01503
Abstract
Providing meaningful and actionable explanations to end-users is a fundamental prerequisite for implementing explainable intelligent systems in the real world. Explainability is a situated interaction between a user and the AI system rather than being static design principles. The content of explanations is context-dependent and must be defined by evidence about the user and its context. This paper seeks to operationalize this concept by proposing a formative architecture that defines the explanation space from a user-inspired perspective. The architecture comprises five intertwined components to outline explanation requirements for a task: (1) the end-users mental models, (2) the end-users cognitive process, (3) the user interface, (4) the human-explainer agent, and the (5) agent process. We first define each component of the architecture. Then we present the Abstracted Explanation Space, a modeling tool that aggregates the architecture's components to support designers in systematically aligning explanations with the end-users work practices, needs, and goals. It guides the specifications of what needs to be explained (content - end-users mental model), why this explanation is necessary (context - end-users cognitive process), to delimit how to explain it (format - human-explainer agent and user interface), and when should the explanations be given. We then exemplify the tool's use in an ongoing case study in the aircraft maintenance domain. Finally, we discuss possible contributions of the tool, known limitations/areas for improvement, and future work to be done.
References in corpus (8)
- Towards A Rigorous Science of Interpretable Machine Learning
- A systematic review and taxonomy of explanations in decision support and recommender systems
- Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
- Metrics for Explainable AI: Challenges and Prospects
- Generating Counterfactual and Contrastive Explanations using SHAP
- Interpreting Classifiers through Attribute Interactions in Datasets
- Principles of Explanation in Human-AI Systems
- A Work-Centered Approach for Cyber-Physical-Social System Design: Applications in Aerospace Industrial Inspection