How to Explain Individual Classification Decisions

Abstract

After building a classifier with modern tools of machine learning we typically have a black box at hand that is able to predict well for unseen data. Thus, we get an answer to the question what is the most likely label of a given unseen data point. However, most methods will provide no answer why the model predicted the particular label for a single instance and what features were most influential for that particular instance. The only method that is currently able to provide such explanations are decision trees. This paper proposes a procedure which (based on a set of assumptions) allows to explain the decisions of any classification method.

31 pages, 14 figures

Cited by in corpus (162)

• Axiomatic Attribution for Deep Networks
• Methods for Interpreting and Understanding Deep Neural Networks
• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• SchNet - a deep learning architecture for molecules and materials
• Interpretable machine learning: definitions, methods, and applications
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• A Survey on the Explainability of Supervised Machine Learning
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• Explaining Explanations in AI
• Towards Explainable Artificial Intelligence
• Consistent Individualized Feature Attribution for Tree Ensembles
• Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision
• "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach
• Explainable AI for Trees: From Local Explanations to Global Understanding
• Explaining the Unique Nature of Individual Gait Patterns with Deep Learning
• Pathologies of Neural Models Make Interpretations Difficult
• Post-hoc Interpretability for Neural NLP: A Survey
• A Deep Learning Interpretable Classifier for Diabetic Retinopathy Disease Grading
• Adversarial attacks and defenses in explainable artificial intelligence: A survey
• Explanations can be manipulated and geometry is to blame
• On the Explainability of Natural Language Processing Deep Models
• Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction
• Toward Explainable AI for Regression Models
• From Clustering to Cluster Explanations via Neural Networks
• WT5?! Training Text-to-Text Models to Explain their Predictions
• Towards Explaining Anomalies: A Deep Taylor Decomposition of One-Class Models
• Hierarchical interpretations for neural network predictions
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Benchmarking Attribution Methods with Relative Feature Importance
• Towards Adversarial Malware Detection: Lessons Learned from PDF-based Attacks
• On the Explanation of Machine Learning Predictions in Clinical Gait Analysis
• An analysis on the use of autoencoders for representation learning: fundamentals, learning task case studies, explainability and challenges
• "Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
• How Important Is a Neuron?
• Probing slow earthquakes with deep learning
• Explainable Machine Learning for Public Policy: Use Cases, Gaps, and Research Directions
• Building and Interpreting Deep Similarity Models
• On the (In)fidelity and Sensitivity for Explanations
• Explainability Techniques for Graph Convolutional Networks
• Restricting the Flow: Information Bottlenecks for Attribution
• Explaining the Explainer: A First Theoretical Analysis of LIME
• The Grammar of Interactive Explanatory Model Analysis
• NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks
• Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking
• Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability
• Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
• A Survey on Neural Network Interpretability
• Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks
• Understanding Integrated Gradients with SmoothTaylor for Deep Neural Network Attribution
• Model Distillation for Revenue Optimization: Interpretable Personalized Pricing
• Towards falsifiable interpretability research
• Shapley Flow: A Graph-based Approach to Interpreting Model Predictions
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Entanglement-guided architectures of machine learning by quantum tensor network
• What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play
• Issues with post-hoc counterfactual explanations: a discussion
• Model extraction from counterfactual explanations
• SLISEMAP: Supervised dimensionality reduction through local explanations
• Explainable $k$-Means and $k$-Medians Clustering
• I-SPLIT: Deep Network Interpretability for Split Computing
• Perceptron Theory Can Predict the Accuracy of Neural Networks
• How Much Can I Trust You? -- Quantifying Uncertainties in Explaining Neural Networks
• Interpreting Super-Resolution Networks with Local Attribution Maps
• Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces
• Towards Robust, Locally Linear Deep Networks
• ML-LOO: Detecting Adversarial Examples with Feature Attribution
• Explaining Bayesian Neural Networks
• Deeply Explain CNN via Hierarchical Decomposition
• Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
• Machine-learning correction to density-functional crystal structure optimization
• Explainable AI guided unsupervised fault diagnostics for high-voltage circuit breakers
• On Network Science and Mutual Information for Explaining Deep Neural Networks
• Why model why? Assessing the strengths and limitations of LIME
• Next2You: Robust Copresence Detection Based on Channel State Information
• Concise Explanations of Neural Networks using Adversarial Training
• Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis
• Spatio-Temporal Perturbations for Video Attribution
• Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features
• Explainability and Graph Learning from Social Interactions
• A Note about: Local Explanation Methods for Deep Neural Networks lack Sensitivity to Parameter Values
• Principal Sensitivity Analysis
• Enhancing the Extraction of Interpretable Information for Ischemic Stroke Imaging from Deep Neural Networks
• Rationalizing Predictions by Adversarial Information Calibration
• Insightful analysis of historical sources at scales beyond human capabilities using unsupervised Machine Learning and XAI
• Provably efficient, succinct, and precise explanations
• Deep Neural Networks for Choice Analysis: A Statistical Learning Theory Perspective
• Computationally Efficient Feature Significance and Importance for Machine Learning Models
• This changes to that : Combining causal and non-causal explanations to generate disease progression in capsule endoscopy
• On the Importance of Domain-specific Explanations in AI-based Cybersecurity Systems (Technical Report)
• XRAI: Better Attributions Through Regions
• Natural Language Interaction with Explainable AI Models
• Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps
• On Tractable Representations of Binary Neural Networks
• The Shapley Taylor Interaction Index
• Debugging Tests for Model Explanations
• Improving Feature Attribution through Input-specific Network Pruning
• Theory-based residual neural networks: A synergy of discrete choice models and deep neural networks
• SISC: End-to-end Interpretable Discovery Radiomics-Driven Lung Cancer Prediction via Stacked Interpretable Sequencing Cells
• Layer-wise Relevance Propagation for Explainable Recommendations
• High Dimensional Model Explanations: an Axiomatic Approach
• Explaining Motion Relevance for Activity Recognition in Video Deep Learning Models
• Optimising for Interpretability: Convolutional Dynamic Alignment Networks
• Deep Neural Networks for Choice Analysis: Extracting Complete Economic Information for Interpretation
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalization
• Explainable AI for ML jet taggers using expert variables and layerwise relevance propagation
• Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition
• Towards Robust Explanations for Deep Neural Networks
• It Takes Two Flints to Make a Fire: Multitask Learning of Neural Relation and Explanation Classifiers
• Toward a Unified Framework for Debugging Concept-based Models
• Attribution in Scale and Space
• The Penalty Imposed by Ablated Data Augmentation
• A Causal Lens for Peeking into Black Box Predictive Models: Predictive Model Interpretation via Causal Attribution
• XInsight: Revealing Model Insights for GNNs with Flow-based Explanations
• Extracting Incentives from Black-Box Decisions
• Large scale evaluation of importance maps in automatic speech recognition
• Shapley Interpretation and Activation in Neural Networks
• Computing Linear Restrictions of Neural Networks
• Model-agnostic interpretation by visualization of feature perturbations
• Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling
• A Comparison of Code Embeddings and Beyond
• Shapley Explanation Networks
• An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
• Crowdsourcing Evaluation of Saliency-based XAI Methods
• Toward the Understanding of Deep Text Matching Models for Information Retrieval
• Improving Interpretability of Deep Neural Networks in Medical Diagnosis by Investigating the Individual Units
• Measuring and improving the quality of visual explanations
• Evaluation of importance estimators in deep learning classifiers for Computed Tomography
• Computing Optimal Decision Sets with SAT
• Fine Grained Dataflow Tracking with Proximal Gradients
• Deep Neural Networks for Choice Analysis: Architectural Design with Alternative-Specific Utility Functions
• New Perspective of Interpretability of Deep Neural Networks
• Unifying Model Explainability and Robustness via Machine-Checkable Concepts
• Do Input Gradients Highlight Discriminative Features?
• Discriminative Attribution from Counterfactuals
• A General Taylor Framework for Unifying and Revisiting Attribution Methods
• Learning by Self-Explanation, with Application to Neural Architecture Search
• A Categorisation of Post-hoc Explanations for Predictive Models
• Understanding of Kernels in CNN Models by Suppressing Irrelevant Visual Features in Images
• ACE -- An Anomaly Contribution Explainer for Cyber-Security Applications
• Assessing the Reliability of Visual Explanations of Deep Models with Adversarial Perturbations
• Model Reduction of Shallow CNN Model for Reliable Deployment of Information Extraction from Medical Reports
• Information-theoretic Evolution of Model Agnostic Global Explanations
• Model-Agnostic Explanations using Minimal Forcing Subsets
• AdjointBackMapV2: Precise Reconstruction of Arbitrary CNN Unit's Activation via Adjoint Operators
• Reinforcement Explanation Learning
• A Framework to Learn with Interpretation
• On Commonsense Cues in BERT for Solving Commonsense Tasks
• Towards Learning a Vocabulary of Visual Concepts and Operators using Deep Neural Networks
• Instance-Based Neural Dependency Parsing
• On Predictive Explanation of Data Anomalies
• Robustness of different loss functions and their impact on networks learning capability
• Towards Auditability for Fairness in Deep Learning
• How to improve the interpretability of kernel learning
• Axiomatic Interpretability for Multiclass Additive Models
• Mutual Information Preserving Back-propagation: Learn to Invert for Faithful Attribution
• Explaining Representation by Mutual Information
• Learning Shape Features and Abstractions in 3D Convolutional Neural Networks for Detecting Alzheimer's Disease
• Towards Interpretable and Learnable Risk Analysis for Entity Resolution
• $\P$ILCRO: Making Importance Landscapes Flat Again