Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysis

Abstract

Explainable artificial intelligence (XAI) is essential for enabling clinical users to get informed decision support from AI and comply with evidence-based medical practice. Applying XAI in clinical settings requires proper evaluation criteria to ensure the explanation technique is both technically sound and clinically useful, but specific support is lacking to achieve this goal. To bridge the research gap, we propose the Clinical XAI Guidelines that consist of five criteria a clinical XAI needs to be optimized for. The guidelines recommend choosing an explanation form based on Guideline 1 (G1) Understandability and G2 Clinical relevance. For the chosen explanation form, its specific XAI technique should be optimized for G3 Truthfulness, G4 Informative plausibility, and G5 Computational efficiency. Following the guidelines, we conducted a systematic evaluation on a novel problem of multi-modal medical image explanation with two clinical tasks, and proposed new evaluation metrics accordingly. Sixteen commonly-used heatmap XAI techniques were evaluated and found to be insufficient for clinical use due to their failure in G3 and G4. Our evaluation demonstrated the use of Clinical XAI Guidelines to support the design and evaluation of clinically viable XAI.

arXiv commentCode: http://github.com/weinajin/multimodal_explanation, Supplementary Material S1 and S2: https://github.com/weinajin/multimodal_explanation/tree/main/paper

References in corpus (14)

• Towards A Rigorous Science of Interpretable Machine Learning
• Striving for Simplicity: The All Convolutional Net
• On Calibration of Modern Neural Networks
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Concept Whitening for Interpretable Image Recognition
• Deep learning cardiac motion analysis for human survival prediction
• Automatic CAD-RADS Scoring Using Deep Learning
• Artificial Intelligence in Glioma Imaging: Challenges and Advances
• AI Research Considerations for Human Existential Safety (ARCHES)
• Synthesis of Brain Tumor MR Images for Learning Data Augmentation
• Adversarial regression training for visualizing the progression of chronic obstructive pulmonary disease with chest x-rays
• EUCA: the End-User-Centered Explainable AI Framework
• One Map Does Not Fit All: Evaluating Saliency Map Explanation on Multi-Modal Medical Images

Cited by in corpus (6)

• Understanding How Blind Users Handle Object Recognition Errors: Strategies and Challenges
• SHAMSUL: Systematic Holistic Analysis to investigate Medical Significance Utilizing Local interpretability methods in deep learning for chest radiography pathology prediction
• Towards a Transparent and Interpretable AI Model for Medical Image Classifications
• A novel framework employing deep multi-attention channels network for the autonomous detection of metastasizing cells through fluorescence microscopy
• CoPA: Hierarchical Concept Prompting and Aggregating Network for Explainable Diagnosis
• M3Net: A Macro-to-Meso-to-Micro Clinical-inspired Hierarchical 3D Network for Pulmonary Nodule Classification