Study on the Helpfulness of Explainable Artificial Intelligence

Abstract

Explainable Artificial Intelligence (XAI) is essential for building advanced machine learning-powered applications, especially in critical domains such as medical diagnostics or autonomous driving. Legal, business, and ethical requirements motivate using effective XAI, but the increasing number of different methods makes it challenging to pick the right ones. Further, as explanations are highly context-dependent, measuring the effectiveness of XAI methods without users can only reveal a limited amount of information, excluding human factors such as the ability to understand it. We propose to evaluate XAI methods via the user's ability to successfully perform a proxy task, designed such that a good performance is an indicator for the explanation to provide helpful information. In other words, we address the helpfulness of XAI for human decision-making. Further, a user study on state-of-the-art methods was conducted, showing differences in their ability to generate trust and skepticism and the ability to judge the rightfulness of an AI decision correctly. Based on the results, we highly recommend using and extending this approach for more objective-based human-centered user studies to measure XAI performance in an end-to-end fashion.

World Conference on Explainable Artificial Intelligence

References in corpus (9)

• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Interpretable machine learning: definitions, methods, and applications
• Shortcut Learning in Deep Neural Networks
• A Survey of Deep Learning Techniques for Autonomous Driving
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences
• From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
• Kandinsky Patterns
• CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations