Metrics for Explainable AI: Challenges and Prospects
arXiv:1812.04608
Abstract
The question addressed in this paper is: If we present to a user an AI system that explains how it works, how do we know whether the explanation works and the user has achieved a pragmatic understanding of the AI? In other words, how do we know that an explanainable AI system (XAI) is any good? Our focus is on the key concepts of measurement. We discuss specific methods for evaluating: (1) the goodness of explanations, (2) whether users are satisfied by explanations, (3) how well users understand the AI systems, (4) how curiosity motivates the search for explanations, (5) whether the user's trust and reliance on the AI are appropriate, and finally, (6) how the human-XAI work system performs. The recommendations we present derive from our integration of extensive research literatures and our own psychometric evaluations.
References in corpus (1)
Cited by in corpus (54)
- A Survey on the Explainability of Supervised Machine Learning
- Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges
- What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
- Explanations in Autonomous Driving: A Survey
- GANterfactual - Counterfactual Explanations for Medical Non-Experts using Generative Adversarial Learning
- AutoDS: Towards Human-Centered Automation of Data Science
- Explainable Activity Recognition for Smart Home Systems
- Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps
- In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes
- An Objective Metric for Explainable AI: How and Why to Estimate the Degree of Explainability
- Causality Learning: A New Perspective for Interpretable Machine Learning
- Getting a CLUE: A Method for Explaining Uncertainty Estimates
- Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act?
- A Survey on Methods and Metrics for the Assessment of Explainability under the Proposed AI Act
- How Much Automation Does a Data Scientist Want?
- A framework for step-wise explaining how to solve constraint satisfaction problems
- Trepan Reloaded: A Knowledge-driven Approach to Explaining Artificial Neural Networks
- Evaluating the Correctness of Explainable AI Algorithms for Classification
- Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
- Improved Explanatory Efficacy on Human Affect and Workload through Interactive Process in Artificial Intelligence
- Quantitative Evaluations on Saliency Methods: An Experimental Study
- Explaining Convolutional Neural Networks through Attribution-Based Input Sampling and Block-Wise Feature Aggregation
- On Relating 'Why?' and 'Why Not?' Explanations
- Trust Considerations for Explainable Robots: A Human Factors Perspective
- Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations
- Don't Explain without Verifying Veracity: An Evaluation of Explainable AI with Video Activity Recognition
- Micro-entries: Encouraging Deeper Evaluation of Mental Models Over Time for Interactive Data Systems
- To Trust or Not to Trust a Regressor: Estimating and Explaining Trustworthiness of Regression Predictions
- Principles of Explanation in Human-AI Systems
- Calibrated Explanations for Regression
- Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations
- Distal Explanations for Model-free Explainable Reinforcement Learning
- X-ToM: Explaining with Theory-of-Mind for Gaining Justified Human Trust
- From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)
- Data Representing Ground-Truth Explanations to Evaluate XAI Methods
- Resisting Out-of-Distribution Data Problem in Perturbation of XAI
- Play MNIST For Me! User Studies on the Effects of Post-Hoc, Example-Based Explanations & Error Rates on Debugging a Deep Learning, Black-Box Classifier
- Explainable AI and Adoption of Financial Algorithmic Advisors: an Experimental Study
- DIVINE: Diverse Influential Training Points for Data Visualization and Model Refinement
- Sequential Explanations with Mental Model-Based Policies
- Towards an Explanation Space to Align Humans and Explainable-AI Teamwork
- Brain-inspired Search Engine Assistant based on Knowledge Graph
- Towards Explainability for a Civilian UAV Fleet Management using an Agent-based Approach
- Iterative Planning with Plan-Space Explanations: A Tool and User Study
- A Comparative Approach to Explainable Artificial Intelligence Methods in Application to High-Dimensional Electronic Health Records: Examining the Usability of XAI
- Explaining Reward Functions to Humans for Better Human-Robot Collaboration
- Critical Empirical Study on Black-box Explanations in AI
- A Multistakeholder Approach Towards Evaluating AI Transparency Mechanisms
- Brittle AI, Causal Confusion, and Bad Mental Models: Challenges and Successes in the XAI Program
- How to Answer Why -- Evaluating the Explanations of AI Through Mental Model Analysis
- Applying Transparency in Artificial Intelligence based Personalization Systems
- Measure Utility, Gain Trust: Practical Advice for XAI Researcher
- XAlgo: a Design Probe of Explaining Algorithms' Internal States via Question-Answering
- Generating User-Centred Explanations via Illocutionary Question Answering: From Philosophy to Interfaces