Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems

Abstract

Several researchers have argued that a machine learning system's interpretability should be defined in relation to a specific agent or task: we should not ask if the system is interpretable, but to whom is it interpretable. We describe a model intended to help answer this question, by identifying different roles that agents can fulfill in relation to the machine learning system. We illustrate the use of our model in a variety of scenarios, exploring how an agent's role influences its goals, and the implications for defining interpretability. Finally, we make suggestions for how our model could be useful to interpretability researchers, system developers, and regulatory bodies auditing machine learning systems.

presented at 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018), Stockholm, Sweden

References in corpus (5)

• Towards A Rigorous Science of Interpretable Machine Learning
• Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
• Attentive Explanations: Justifying Decisions and Pointing to the Evidence
• TIP: Typifying the Interpretability of Procedures
• Enhancing Transparency and Control when Drawing Data-Driven Inferences about Individuals

Cited by in corpus (17)

• Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
• Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring
• Stakeholders in Explainable AI
• Explainable Artificial Intelligence (XAI): An Engineering Perspective
• Techniques for Interpretable Machine Learning
• Question-Driven Design Process for Explainable AI User Experiences
• Fast Hierarchical Games for Image Explanations
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience
• Personalized explanation in machine learning: A conceptualization
• Explainable AI for Intelligence Augmentation in Multi-Domain Operations
• Hows and Whys of Artificial Intelligence for Public Sector Decisions: Explanation and Evaluation
• What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversations
• Situated Case Studies for a Human-Centered Design of Explanation User Interfaces
• Seven challenges for harmonizing explainability requirements
• Reasons, Values, Stakeholders: A Philosophical Framework for Explainable Artificial Intelligence
• On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System
• Towards human-agent knowledge fusion (HAKF) in support of distributed coalition teams