A Survey on the Explainability of Supervised Machine Learning

Abstract

Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or fifinance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Learning (SML). We conduct a state-of-the-art survey that reviews past and recent explainable SML approaches and classifies them according to the introduced definitions. Finally, we illustrate principles by means of an explanatory case study and discuss important future directions.

Accepted for publication at the Journal of Artificial Intelligence Research (JAIR)

References in corpus (77)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• A Unified Approach to Interpreting Model Predictions
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Towards A Rigorous Science of Interpretable Machine Learning
• Methods for Interpreting and Understanding Deep Neural Networks
• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• Interpretable machine learning: definitions, methods, and applications
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Understanding Black-box Predictions via Influence Functions
• Predictive learning via rule ensembles
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• A Survey on the Explainability of Supervised Machine Learning
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• SmoothGrad: removing noise by adding noise
• How to Explain Individual Classification Decisions
• Model-Agnostic Interpretability of Machine Learning
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Attention is not Explanation
• Towards Robust Interpretability with Self-Explaining Neural Networks
• What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
• Supersparse Linear Integer Models for Optimized Medical Scoring Systems
• FACE: Feasible and Actionable Counterfactual Explanations
• Distilling a Neural Network Into a Soft Decision Tree
• Prototype selection for interpretable classification
• Metrics for Explainable AI: Challenges and Prospects
• Local Rule-Based Explanations of Black Box Decision Systems
• Modeling Missing Data in Clinical Time Series with RNNs
• Interpreting Blackbox Models via Model Extraction
• Model Agnostic Supervised Local Explanations
• An Evaluation of the Human-Interpretability of Explanation
• The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
• Node harvest
• NeuroRule: A Connectionist Approach to Data Mining
• Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
• Deep Neural Decision Trees
• Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers
• Gradients of Counterfactuals
• Quantifying Interpretability and Trust in Machine Learning Systems
• Making Tree Ensembles Interpretable
• The Doctor Just Won't Accept That!
• Attentive Explanations: Justifying Decisions and Pointing to the Evidence
• Inverse Classification for Comparison-based Interpretability in Machine Learning
• Defining Locality for Surrogates in Post-hoc Interpretablity
• The Promise and Peril of Human Evaluation for Model Interpretability
• Distribution-Free Predictive Inference For Regression
• Interpreting Classifiers through Attribute Interactions in Datasets
• Interpretable & Explorable Approximations of Black Box Models
• Methods and Models for Interpretable Linear Classification
• Knowledge-based Transfer Learning Explanation
• Explaining Trained Neural Networks with Semantic Web Technologies: First Steps
• Or's of And's for Interpretable Classification, with Application to Context-Aware Recommender Systems
• Interpretable Two-level Boolean Rule Learning for Classification
• Explainable Restricted Boltzmann Machines for Collaborative Filtering
• Global Model Interpretation via Recursive Partitioning
• Triaging moderate COVID-19 and other viral pneumonias from routine blood tests
• Neural Decision Trees
• Interpretable Two-level Boolean Rule Learning for Classification
• ML-Schema: Exposing the Semantics of Machine Learning with Schemas and Ontologies
• Classification by Set Cover: The Prototype Vector Machine
• Towards Aggregating Weighted Feature Attributions
• Scalable Bayesian Rule Lists
• An Investigation of COVID-19 Spreading Factors with Explainable AI Techniques
• Enhancing Transparency and Control when Drawing Data-Driven Inferences about Individuals
• Human-centric Transfer Learning Explanation via Knowledge Graph [Extended Abstract]
• REx: An Efficient Rule Generator

Cited by in corpus (62)

• A Survey on the Explainability of Supervised Machine Learning
• From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
• On scientific understanding with artificial intelligence
• A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
• A Clarification of the Nuances in the Fairness Metrics Landscape
• Choice modelling in the age of machine learning -- discussion paper
• Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review
• Using Sequences of Life-events to Predict Human Lives
• Survey on Machine Learning for Traffic-Driven Service Provisioning in Optical Networks
• The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations
• Training Data Influence Analysis and Estimation: A Survey
• A perspective on protein structure prediction using quantum computers
• Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act?
• Predicting Good Quantum Circuit Compilation Options
• Explainable Machine Learning with Prior Knowledge: An Overview
• Predictive Compliance Monitoring in Process-Aware Information Systems: State of the Art, Functionalities, Research Directions
• Human Interpretation of Saliency-based Explanation Over Text
• Interpretable Clustering: A Survey
• Fairness in Agreement With European Values: An Interdisciplinary Perspective on AI Regulation
• The Shapley Value in Database Management
• Identifying Student Profiles Within Online Judge Systems Using Explainable Artificial Intelligence
• Explainable automatic industrial carbon footprint estimation from bank transaction classification using natural language processing
• Explainable AI using expressive Boolean formulas
• Symbolic Regression as Feature Engineering Method for Machine and Deep Learning Regression Tasks
• XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning
• Unboxing Tree Ensembles for interpretability: a hierarchical visualization tool and a multivariate optimal re-built tree
• Postprocessing of Ensemble Weather Forecasts Using Permutation-invariant Neural Networks
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box
• Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing
• Achieving Diversity in Counterfactual Explanations: a Review and Discussion
• Interpreting the Mechanism of Synergism for Drug Combinations Using Attention-Based Hierarchical Graph Pooling
• Explainable cognitive decline detection in free dialogues with a Machine Learning approach based on pre-trained Large Language Models
• Assessment of the influence of features on a classification problem: an application to COVID-19 patients
• A Comprehensive Survey on Self-Interpretable Neural Networks
• Accurate Passive Radar via an Uncertainty-Aware Fusion of Wi-Fi Sensing Data
• A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models
• FeBiM: Efficient and Compact Bayesian Inference Engine Empowered with Ferroelectric In-Memory Computing
• A Survey on Event Prediction Methods from a Systems Perspective: Bringing Together Disparate Research Areas
• Promoting Security and Trust on Social Networks: Explainable Cyberbullying Detection Using Large Language Models in a Stream-Based Machine Learning Framework
• Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces
• Comparison of generalised additive models and neural networks in applications: A systematic review
• Automatic generation of insights from workers' actions in industrial workflows with explainable Machine Learning
• Scene Text Recognition Models Explainability Using Local Features
• Decoding the Multimodal Maze: A Systematic Review on the Adoption of Explainability in Multimodal Attention-based Models
• Deep Quantum Graph Dreaming: Deciphering Neural Network Insights into Quantum Experiments
• Concept Extraction for Time Series with ECLAD-ts
• Quantifying Process Quality: The Role of Effective Organizational Learning in Software Evolution
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
• Utilizing dataset affinity prediction in object detection to assess training data
• The Impact of Machine Learning Uncertainty on the Robustness of Counterfactual Explanations
• Efficient Decompositional Rule Extraction for Deep Neural Networks
• Interpretability is not Explainability: New Quantitative XAI Approach with a focus on Recommender Systems in Education
• Explanation format does not matter; but explanations do -- An Eggsbert study on explaining Bayesian Optimisation tasks
• Simulating counterfactuals
• Sample Observed Effects: Enumeration, Randomization and Generalization
• FREQuency ATTribution: benchmarking frequency-based occlusion for time series data
• Towards Explainable Scientific Venue Recommendations
• Conceptualizing Uncertainty: A Concept-based Approach to Explaining Uncertainty
• Deep Learning-Based Detection of the Acute Respiratory Distress Syndrome: What Are the Models Learning?
• Explainability-Aware One Point Attack for Point Cloud Neural Networks