Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

Abstract

With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outlook of future opportunities for XAI in medical image analysis.

Submitted for publication. Comments welcome by email to first author

References in corpus (37)

• A Survey on Deep Learning in Medical Image Analysis
• Towards A Rigorous Science of Interpretable Machine Learning
• Interpretable machine learning: definitions, methods, and applications
• Understanding Black-box Predictions via Influence Functions
• Open Source Software for Efficient and Transparent Reviews
• Sanity Checks for Saliency Maps
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Causality matters in medical imaging
• Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification
• Uncertainty and Interpretability in Convolutional Neural Networks for Semantic Segmentation of Colorectal Polyps
• Going Deeper through the Gleason Scoring Scale: An Automatic end-to-end System for Histology Prostate Grading and Cribriform Pattern Detection
• Resolving challenges in deep learning-based analyses of histopathological images using explanation methods
• Computer-aided diagnosis in histopathological images of the endometrium using a convolutional neural network and attention mechanisms
• DR$|$GRADUATE: uncertainty-aware deep learning-based diabetic retinopathy grading in eye fundus images
• Direct Automatic Coronary Calcium Scoring in Cardiac and Chest CT
• Learning to detect chest radiographs containing lung nodules using visual attention networks
• SDFN: Segmentation-based Deep Fusion Network for Thoracic Disease Classification in Chest X-ray Images
• Melanoma Detection using Adversarial Training and Deep Transfer Learning
• Decoding and mapping task states of the human brain via deep learning
• Shape and Margin-Aware Lung Nodule Classification in Low-dose CT Images via Soft Activation Mapping
• Dynamic Routing Between Capsules
• Automatic brain tumor grading from MRI data using convolutional neural networks and quality assessment
• Deep neural network or dermatologist?
• HMIC: Hierarchical Medical Image Classification, A Deep Learning Approach
• Automated Coronary Artery Atherosclerosis Detection and Weakly Supervised Localization on Coronary CT Angiography with a Deep 3-Dimensional Convolutional Neural Network
• Neural Network Attributions: A Causal Perspective
• Intrapapillary Capillary Loop Classification in Magnification Endoscopy: Open Dataset and Baseline Methodology
• Regularizing Neural Network Training via Identity-wise Discriminative Feature Suppression
• Regional Multi-scale Approach for Visually Pleasing Explanations of Deep Neural Networks
• Controlling for Biasing Signals in Images for Prognostic Models: Survival Predictions for Lung Cancer with Deep Learning
• Structural Compression of Convolutional Neural Networks
• Justifying Diagnosis Decisions by Deep Neural Networks
• Looking in the Right place for Anomalies: Explainable AI through Automatic Location Learning
• Automated Enriched Medical Concept Generation for Chest X-ray Images
• CORAL8: Concurrent Object Regression for Area Localization in Medical Image Panels
• Deep Angular Embedding and Feature Correlation Attention for Breast MRI Cancer Analysis
• Thoracic Disease Identification and Localization with Limited Supervision

Cited by in corpus (32)

• Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions
• Memory-aware curriculum federated learning for breast cancer classification
• Automated Detection and Forecasting of COVID-19 using Deep Learning Techniques: A Review
• Domain Generalization for Medical Image Analysis: A Review
• A scoping review on multimodal deep learning in biomedical images and texts
• Understanding Calibration of Deep Neural Networks for Medical Image Classification
• sMRI-PatchNet: A novel explainable patch-based deep learning network for Alzheimer's disease diagnosis and discriminative atrophy localisation with Structural MRI
• Weakly-supervised detection of AMD-related lesions in color fundus images using explainable deep learning
• Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretation
• Transfer Learning for the Efficient Detection of COVID-19 from Smartphone Audio Data
• Unveiling the Human-like Similarities of Automatic Facial Expression Recognition: An Empirical Exploration through Explainable AI
• Cloud-based XAI Services for Assessing Open Repository Models Under Adversarial Attacks
• Clinical Domain Knowledge-Derived Template Improves Post Hoc AI Explanations in Pneumothorax Classification
• Towards consistency of rule-based explainer and black box model -- fusion of rule induction and XAI-based feature importance
• Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features
• IODeep: an IOD for the introduction of deep learning in the DICOM standard
• EvalAttAI: A Holistic Approach to Evaluating Attribution Maps in Robust and Non-Robust Models
• Robust and Explainable Framework to Address Data Scarcity in Diagnostic Imaging
• The Explanation Necessity for Healthcare AI
• Towards a Transparent and Interpretable AI Model for Medical Image Classifications
• Understanding whole-body inter-personal dynamics between two players using neural Granger causality as the explainable AI (XAI)
• PhagoStat a scalable and interpretable end to end framework for efficient quantification of cell phagocytosis in neurodegenerative disease studies
• Effective Predictive Modeling for Emergency Department Visits and Evaluating Exogenous Variables Impact: Using Explainable Meta-learning Gradient Boosting
• Reducing Annotation Need in Self-Explanatory Models for Lung Nodule Diagnosis
• Meta-evaluating stability measures: MAX-Senstivity & AVG-Sensitivity
• Learning Interpretable Microscopic Features of Tumor by Multi-task Adversarial CNNs To Improve Generalization
• Intuitive Axial Augmentation Using Polar-Sine-Based Piecewise Distortion for Medical Slice-Wise Segmentation
• WEEP: A method for spatial interpretation of weakly supervised CNN models in computational pathology
• A novel framework employing deep multi-attention channels network for the autonomous detection of metastasizing cells through fluorescence microscopy
• Federated Explainable Artificial Intelligence: Roles, Architectures, Evaluation, and Open Challenges
• CoPA: Hierarchical Concept Prompting and Aggregating Network for Explainable Diagnosis
• An explainable three dimension framework to uncover learning patterns: A unified look in variable sulci recognition