A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

Abstract

Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning. Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand the mechanism underlying the algorithms. Unfortunately, the blackbox nature of the deep learning is still unresolved, and many machine decisions are still poorly understood. We provide a review on interpretabilities suggested by different research works and categorize them. The different categories show different dimensions in interpretability research, from approaches that provide "obviously" interpretable information to the studies of complex patterns. By applying the same categorization to interpretability in medical research, it is hoped that (1) clinicians and practitioners can subsequently approach these methods with caution, (2) insights into interpretability will be born with more considerations for medical practices, and (3) initiatives to push forward data-based, mathematically- and technically-grounded medical education are encouraged.

References in corpus (46)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Methods for Interpreting and Understanding Deep Neural Networks
• Striving for Simplicity: The All Convolutional Net
• Learning Important Features Through Propagating Activation Differences
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Understanding Neural Networks Through Deep Visualization
• Understanding Black-box Predictions via Influence Functions
• Hierarchical Question-Image Co-Attention for Visual Question Answering
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• Interpretable Explanations of Black Boxes by Meaningful Perturbation
• Visualizing and Understanding Recurrent Networks
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Opening the Black Box of Deep Neural Networks via Information
• SmoothGrad: removing noise by adding noise
• How to Explain Individual Classification Decisions
• Detecting Cancer Metastases on Gigapixel Pathology Images
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Learning how to explain neural networks: PatternNet and PatternAttribution
• "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach
• What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
• SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
• Ablation Studies in Artificial Neural Networks
• The (Un)reliability of saliency methods
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
• Brain Age from the Electroencephalogram of Sleep
• Explanations can be manipulated and geometry is to blame
• An Evaluation of the Human-Interpretability of Explanation
• Deep neural network or dermatologist?
• Interpretation of Neural Networks is Fragile
• Representer Point Selection for Explaining Deep Neural Networks
• Building and Interpreting Deep Similarity Models
• Weakly Supervised Instance Segmentation using Class Peak Response
• Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation
• Autofocus Layer for Semantic Segmentation
• Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations
• Inferring Robot Task Plans from Human Team Meetings: A Generative Modeling Approach with Logic-Based Prior
• Understanding the (un)interpretability of natural image distributions using generative models
• A Large-scale Multimodal Study for Predicting Mortality Risk Using Minimal and Low Parameter Models and Separable Risk Assessment
• Generative discriminative models for multivariate inference and statistical mapping in medical imaging
• Explaining Motion Relevance for Activity Recognition in Video Deep Learning Models
• Interpretable Dynamics Models for Data-Efficient Reinforcement Learning

Cited by in corpus (130)

• Deep Neural Networks and Tabular Data: A Survey
• A Survey on the Explainability of Supervised Machine Learning
• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
• Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
• On scientific understanding with artificial intelligence
• A Survey on ChatGPT: AI-Generated Contents, Challenges, and Solutions
• Explanations in Autonomous Driving: A Survey
• A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts
• A Survey on Digital Twins: Architecture, Enabling Technologies, Security and Privacy, and Future Prospects
• Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
• Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions
• A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
• Designing Interpretable ML System to Enhance Trust in Healthcare: A Systematic Review to Proposed Responsible Clinician-AI-Collaboration Framework
• Post-hoc Interpretability for Neural NLP: A Survey
• Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence
• Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions
• Explainable Artificial Intelligence: Precepts, Methods, and Opportunities for Research in Construction
• Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making
• A Survey on Explainable Artificial Intelligence for Cybersecurity
• Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
• Deep Learning for Computational Cytology: A Survey
• A Survey on AI Assurance
• Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective
• Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset
• Development and evaluation of an Explainable Prediction Model for Chronic Kidney Disease Patients based on Ensemble Trees
• Explainable Artificial Intelligence Approaches: A Survey
• From Modern CNNs to Vision Transformers: Assessing the Performance, Robustness, and Classification Strategies of Deep Learning Models in Histopathology
• A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directions
• CLIP in Medical Imaging: A Survey
• Explainable AI: current status and future directions
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms
• Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
• Explainable Activity Recognition for Smart Home Systems
• Improvement of a Prediction Model for Heart Failure Survival through Explainable Artificial Intelligence
• A scoping review on multimodal deep learning in biomedical images and texts
• Rationalization for Explainable NLP: A Survey
• Neurosymbolic AI for Reasoning over Knowledge Graphs: A Survey
• On Tackling Explanation Redundancy in Decision Trees
• Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations
• Application of artificial intelligence techniques for automated detection of myocardial infarction: A review
• Explainable Artificial Intelligence (XAI): An Engineering Perspective
• Can autism be diagnosed with AI?
• AI Human Impact: Toward a Model for Ethical Investing in AI-Intensive Companies
• Socially Responsible AI Algorithms: Issues, Purposes, and Challenges
• A Comprehensive Survey of Machine Learning Applied to Radar Signal Processing
• Explainable Knowledge Distillation for On-device Chest X-Ray Classification
• Multimodal Explainability via Latent Shift applied to COVID-19 stratification
• Explaining Human Activity Recognition with SHAP: Validating Insights with Perturbation and Quantitative Measures
• Human Interpretation of Saliency-based Explanation Over Text
• Explaining Time Series Predictions with Dynamic Masks
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Parkinson's Disease Diagnosis based on Gait Cycle Analysis Through an Interpretable Interval Type-2 Neuro-Fuzzy System
• An Explainable Deep Learning-Based Method For Schizophrenia Diagnosis Using Generative Data-Augmentation
• Improving the Timing Resolution of Positron Emission Tomography Detectors Using Boosted Learning -- A Residual Physics Approach
• Insights Into the Inner Workings of Transformer Models for Protein Function Prediction
• Artificial intelligence and radiation protection. A game changer or an update?
• Coalitional strategies for efficient individual prediction explanation
• Path-based Explanation for Knowledge Graph Completion
• CogniAlign: Word-Level Multimodal Speech Alignment with Gated Cross-Attention for Alzheimer's Detection
• Towards Real-World Applications of Personalized Anesthesia Using Policy Constraint Q Learning for Propofol Infusion Control
• Explaining Bayesian Neural Networks
• Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers
• Achieving Diversity in Counterfactual Explanations: a Review and Discussion
• Learning at a Glance: Towards Interpretable Data-limited Continual Semantic Segmentation via Semantic-Invariance Modelling
• Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond
• GCExplainer: Human-in-the-Loop Concept-based Explanations for Graph Neural Networks
• On quantitative aspects of model interpretability
• Black-box Error Diagnosis in Deep Neural Networks for Computer Vision: a Survey of Tools
• Explainable AI (XAI) for PHM of Industrial Asset: A State-of-The-Art, PRISMA-Compliant Systematic Review
• Feature construction using explanations of individual predictions
• Efficient computation of counterfactual explanations of LVQ models
• Choose Your Explanation: A Comparison of SHAP and GradCAM in Human Activity Recognition
• Clinical information extraction for Low-resource languages with Few-shot learning using Pre-trained language models and Prompting
• Discovering robust biomarkers of psychiatric disorders from resting-state functional MRI via graph neural networks: A systematic review
• Enhancing the Extraction of Interpretable Information for Ischemic Stroke Imaging from Deep Neural Networks
• Insightful analysis of historical sources at scales beyond human capabilities using unsupervised Machine Learning and XAI
• Inferring ECG from PPG for Continuous Cardiac Monitoring Using Lightweight Neural Network
• Multi-Task Learning for Post-transplant Cause of Death Analysis: A Case Study on Liver Transplant
• Saliency-Bench: A Comprehensive Benchmark for Evaluating Visual Explanations
• MARLens: Understanding Multi-agent Reinforcement Learning for Traffic Signal Control via Visual Analytics
• Explaining Deep Learning Models for Structured Data using Layer-Wise Relevance Propagation
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• MeLIME: Meaningful Local Explanation for Machine Learning Models
• Out-of-Distribution Generalization in Time Series: A Survey
• Explaining Predictions by Approximating the Local Decision Boundary
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• Technical Understanding from IML Hands-on Experience: A Study through a Public Event for Science Museum Visitors
• Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming
• Techniques for Symbol Grounding with SATNet
• Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks
• Integrating Chain-of-Thought and Retrieval Augmented Generation Enhances Rare Disease Diagnosis from Clinical Notes
• Beyond the Hype: A dispassionate look at vision-language models in medical scenario
• Utilizing Explainable AI for Quantization and Pruning of Deep Neural Networks
• SurvLIME-Inf: A simplified modification of SurvLIME for explanation of machine learning survival models
• Neural Generators of Sparse Local Linear Models for Achieving both Accuracy and Interpretability
• Towards Optimising EEG Decoding using Post-hoc Explanations and Domain Knowledge
• Interpretable Models Capable of Handling Systematic Missingness in Imbalanced Classes and Heterogeneous Datasets
• ACTA: A Mobile-Health Solution for Integrated Nudge-Neurofeedback Training for Senior Citizens
• EnTri: Ensemble Learning with Tri-level Representations for Explainable Scene Recognition
• Understanding whole-body inter-personal dynamics between two players using neural Granger causality as the explainable AI (XAI)
• ExplainFix: Explainable Spatially Fixed Deep Networks
• On Explainability in AI-Solutions: A Cross-Domain Survey
• Reducing Annotation Need in Self-Explanatory Models for Lung Nodule Diagnosis
• Reliable Graph Neural Network Explanations Through Adversarial Training
• POTHER: Patch-Voted Deep Learning-Based Chest X-ray Bias Analysis for COVID-19 Detection
• You Can Do Better! If You Elaborate the Reason When Making Prediction
• Deep Quantum Graph Dreaming: Deciphering Neural Network Insights into Quantum Experiments
• Explainable AI for computational pathology identifies model limitations and tissue biomarkers
• Generalization on the Enhancement of Layerwise Relevance Interpretability of Deep Neural Network
• Simplifying the explanation of deep neural networks with sufficient and necessary feature-sets: case of text classification
• Stealing Black-Box Functionality Using The Deep Neural Tree Architecture
• A mathematical certification for positivity conditions in Neural Networks with applications to partial monotonicity and Trustworthy AI
• Causality-Driven Neural Network Repair: Challenges and Opportunities
• A novel framework employing deep multi-attention channels network for the autonomous detection of metastasizing cells through fluorescence microscopy
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
• Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration
• Finding Words Associated with DIF: Predicting Differential Item Functioning using LLMs and Explainable AI
• Scaling up Memory-Efficient Formal Verification Tools for Tree Ensembles
• Corpus-level and Concept-based Explanations for Interpretable Document Classification
• BI-RADS-Net: An Explainable Multitask Learning Approach for Cancer Diagnosis in Breast Ultrasound Images
• Color Teams for Machine Learning Development
• Use of Parallel Explanatory Models to Enhance Transparency of Neural Network Configurations for Cell Degradation Detection
• Learning Efficient, Explainable and Discriminative Representations for Pulmonary Nodules Classification
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
• Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition
• Predicting and Explaining Hearing Aid Usage Using Encoder-Decoder with Attention Mechanism and SHAP
• Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical Approach
• An Incremental MaxSAT-based Model to Learn Interpretable and Balanced Classification Rules