Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

Abstract

Nowadays, deep neural networks are widely used in mission critical systems such as healthcare, self-driving vehicles, and military which have direct impact on human lives. However, the black-box nature of deep neural networks challenges its use in mission critical applications, raising ethical and judicial concerns inducing lack of trust. Explainable Artificial Intelligence (XAI) is a field of Artificial Intelligence (AI) that promotes a set of tools, techniques, and algorithms that can generate high-quality interpretable, intuitive, human-understandable explanations of AI decisions. In addition to providing a holistic view of the current XAI landscape in deep learning, this paper provides mathematical summaries of seminal work. We start by proposing a taxonomy and categorizing the XAI techniques based on their scope of explanations, methodology behind the algorithms, and explanation level or usage which helps build trustworthy, interpretable, and self-explanatory deep learning models. We then describe the main principles used in XAI research and present the historical timeline for landmark studies in XAI from 2007 to 2020. After explaining each category of algorithms and approaches in detail, we then evaluate the explanation maps generated by eight XAI algorithms on image data, discuss the limitations of this approach, and provide potential future directions to improve XAI evaluation.

arXiv comment24 pages, 20 figures, survey paper, submitting to IEEE

References in corpus (27)

• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Learning Important Features Through Propagating Activation Differences
• A Survey of Deep Learning Techniques for Autonomous Driving
• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Sanity Checks for Saliency Maps
• Towards Robust Interpretability with Self-Explaining Neural Networks
• Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations
• What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
• Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning
• RISE: Randomized Input Sampling for Explanation of Black-box Models
• Neural Additive Models: Interpretable Machine Learning with Neural Nets
• Explaining Classifiers with Causal Concept Effect (CaCE)
• Benchmarking Attribution Methods with Relative Feature Importance
• Explaining Anomalies Detected by Autoencoders Using SHAP
• Adversarial Attacks on Neural Network Policies
• On Completeness-aware Concept-Based Explanations in Deep Neural Networks
• Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation
• A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
• Human Action Performance using Deep Neuro-Fuzzy Recurrent Attention Model
• Interpreting Black Box Models via Hypothesis Testing
• Local Interpretable Model-agnostic Explanations of Bayesian Predictive Models via Kullback-Leibler Projections
• A Human-Grounded Evaluation of SHAP for Alert Processing
• A Modified Perturbed Sampling Method for Local Interpretable Model-agnostic Explanation

Cited by in corpus (63)

• Explanations in Autonomous Driving: A Survey
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
• Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
• A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
• Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions
• A Survey on Explainable Artificial Intelligence for Cybersecurity
• Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
• On the Explainability of Natural Language Processing Deep Models
• A Survey on AI Assurance
• Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
• Investigating and Designing for Trust in AI-powered Code Generation Tools
• Explainable AI: current status and future directions
• Logic Explained Networks
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
• Explainable Activity Recognition for Smart Home Systems
• Entropy-based Logic Explanations of Neural Networks
• Explainable Artificial Intelligence for Pharmacovigilance: What Features Are Important When Predicting Adverse Outcomes?
• Toward Accurate Interpretable Predictions of Materials Properties within Transformer Language Models
• AI-Augmented Behavior Analysis for Children with Developmental Disabilities: Building Towards Precision Treatment
• Explainable Artificial Intelligence (XAI): An Engineering Perspective
• A Survey on Privacy for B5G/6G: New Privacy Challenges, and Research Directions
• Learning from Few Samples: A Survey
• AI Human Impact: Toward a Model for Ethical Investing in AI-Intensive Companies
• sMRI-PatchNet: A novel explainable patch-based deep learning network for Alzheimer's disease diagnosis and discriminative atrophy localisation with Structural MRI
• Path To Gain Functional Transparency In Artificial Intelligence With Meaningful Explainability
• Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey
• Explaining Time Series Predictions with Dynamic Masks
• Privacy Meets Explainability: A Comprehensive Impact Benchmark
• Evaluating the Effectiveness of XAI Techniques for Encoder-Based Language Models
• Calibrated Explanations: with Uncertainty Information and Counterfactuals
• Evaluating the Correctness of Explainable AI Algorithms for Classification
• Predicting Events in MOBA Games: Prediction, Attribution, and Evaluation
• Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
• Future Artificial Intelligence tools and perspectives in medicine
• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• Explaining Bayesian Neural Networks
• ExplainitAI: When do we trust artificial intelligence? The influence of content and explainability in a cross-cultural comparison
• Benchmarking the Influence of Pre-training on Explanation Performance in MR Image Classification
• Visualization Of Class Activation Maps To Explain AI Classification Of Network Packet Captures
• DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction
• The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research
• Technical Understanding from IML Hands-on Experience: A Study through a Public Event for Science Museum Visitors
• The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples
• Explainable Goal-Driven Agents and Robots -- A Comprehensive Review
• Reward Bonuses with Gain Scheduling Inspired by Iterative Deepening Search
• Methodology and Real-World Applications of Dynamic Uncertain Causality Graph for Clinical Diagnosis with Explainability and Invariance
• Stuttering Speech Disfluency Prediction using Explainable Attribution Vectors of Facial Muscle Movements
• PyTorch, Explain! A Python library for Logic Explained Networks
• EnTri: Ensemble Learning with Tri-level Representations for Explainable Scene Recognition
• Zero-bias Deep Learning Enabled Quick and Reliable Abnormality Detection in IoT
• Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testing
• On the Definition of Appropriate Trust and the Tools that Come with it
• Sampling - Variational Auto Encoder - Ensemble: In the Quest of Explainable Artificial Intelligence
• Scope and Sense of Explainability for AI-Systems
• Ensembles of Random SHAPs
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
• Corpus-level and Concept-based Explanations for Interpretable Document Classification
• Understanding the Dynamics of DNNs Using Graph Modularity
• Image recognition via Vietoris-Rips complex
• Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition
• Nose to Glass: Looking In to Get Beyond