Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations

Abstract

Recent success in Artificial Intelligence (AI) and Machine Learning (ML) allow problem solving automatically without any human intervention. Autonomous approaches can be very convenient. However, in certain domains, e.g., in the medical domain, it is necessary to enable a domain expert to understand, why an algorithm came up with a certain result. Consequently, the field of Explainable AI (xAI) rapidly gained interest worldwide in various domains, particularly in medicine. Explainable AI studies transparency and traceability of opaque AI/ML and there are already a huge variety of methods. For example with layer-wise relevance propagation relevant parts of inputs to, and representations in, a neural network which caused a result, can be highlighted. This is a first important step to ensure that end users, e.g., medical professionals, assume responsibility for decision making with AI/ML and of interest to professionals and regulators. Interactive ML adds the component of human expertise to AI/ML processes by enabling them to re-enact and retrace AI/ML results, e.g. let them check it for plausibility. This requires new human-AI interfaces for explainable AI. In order to build effective and efficient interactive human-AI interfaces we have to deal with the question of how to evaluate the quality of explanations given by an explainable AI system. In this paper we introduce our System Causability Scale (SCS) to measure the quality of explanations. It is based on our notion of Causability (Holzinger et al., 2019) combined with concepts adapted from a widely accepted usability scale.

arXiv comment6 pages, 1 figure, 1 table, will appear in Springer/Nature KI - Künstliche Intelligenz (2020), Volume 34, Issue 2

References in corpus (3)

• A Survey on Deep Learning in Medical Image Analysis
• Towards A Rigorous Science of Interpretable Machine Learning
• Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge

Cited by in corpus (41)

• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
• Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI
• Explainable Deep Reinforcement Learning: State of the Art and Challenges
• Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective
• AI, Opacity, and Personal Autonomy
• Explainable Machine Learning for Public Policy: Use Cases, Gaps, and Research Directions
• An Objective Metric for Explainable AI: How and Why to Estimate the Degree of Explainability
• Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
• Survey of XAI in digital pathology
• A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoption
• How well can a large language model explain business processes as perceived by users?
• Kandinsky Patterns
• A Survey on Methods and Metrics for the Assessment of Explainability under the Proposed AI Act
• Adequate and fair explanations
• Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting
• A Meta Survey of Quality Evaluation Criteria in Explanation Methods
• Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learning
• Embedding Deep Networks into Visual Explanations
• A Conceptual Framework for Externally-influenced Agents: An Assisted Reinforcement Learning Review
• What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks
• Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
• How Can We Develop Explainable Systems? Insights from a Literature Review and an Interview Study
• Using ChatGPT in HCI Research -- A Trioethnography
• Simulation of Multi-Stage Attack and Defense Mechanisms in Smart Grids
• Explainable AI (XAI) for PHM of Industrial Asset: A State-of-The-Art, PRISMA-Compliant Systematic Review
• APOLLO: A GPT-based tool to detect phishing emails and generate explanations that warn users
• Towards Interpretability in Audio and Visual Affective Machine Learning: A Review
• FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging
• Optimising Knee Injury Detection with Spatial Attention and Validating Localisation Ability
• On Specification-based Cyber-Attack Detection in Smart Grids
• For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAI
• XAI-CF -- Examining the Role of Explainable Artificial Intelligence in Cyber Forensics
• Resisting Out-of-Distribution Data Problem in Perturbation of XAI
• Assessing AI Explainability: A Usability Study Using a Novel Framework Involving Clinicians
• On the Definition of Appropriate Trust and the Tools that Come with it
• Explanatory Pluralism in Explainable AI
• How to Answer Why -- Evaluating the Explanations of AI Through Mental Model Analysis
• Understanding User Mental Models in AI-Driven Code Completion Tools: Insights from an Elicitation Study
• A Study into patient similarity through representation learning from medical records
• The Impact of Concept Explanations and Interventions on Human-Machine Collaboration