Interpretations are useful: penalizing explanations to align neural networks with prior knowledge

Abstract

For an explanation of a deep learning model to be effective, it must provide both insight into a model and suggest a corresponding action in order to achieve some objective. Too often, the litany of proposed explainable deep learning methods stop at the first step, providing practitioners with insight into a model, but no way to act on it. In this paper, we propose contextual decomposition explanation penalization (CDEP), a method which enables practitioners to leverage existing explanation methods in order to increase the predictive accuracy of deep learning models. In particular, when shown that a model has incorrectly assigned importance to some features, CDEP enables practitioners to correct these errors by directly regularizing the provided explanations. Using explanations provided by contextual decomposition (CD) (Murdoch et al., 2018), we demonstrate the ability of our method to increase performance on an array of toy and real datasets.

arXiv comment18 pages; published in ICML2020; Erratum: numbers in table 1 were too high (now corrected) with the trend remaining the same

References in corpus (26)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• Interpretable machine learning: definitions, methods, and applications
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• How to Explain Individual Classification Decisions
• Sanity Checks for Saliency Maps
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Attention is not Explanation
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
• Women also Snowboard: Overcoming Bias in Captioning Models
• Hierarchical interpretations for neural network predictions
• Real Time Image Saliency for Black Box Classifiers
• Automatic Rule Extraction from Long Short Term Memory Networks
• Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer
• A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
• Transformation Importance with Applications to Cosmology
• Incorporating Priors with Feature Attribution on Text Classification
• Deriving Machine Attention from Human Rationales
• Learning Credible Deep Neural Networks with Rationale Regularization
• Aggregating explanation methods for stable and robust explainability

Cited by in corpus (12)

• Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning
• Saliency is a Possible Red Herring When Diagnosing Poor Generalization
• Transformation Importance with Applications to Cosmology
• Trustworthy Convolutional Neural Networks: A Gradient Penalized-based Approach
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• Learning Deep Attribution Priors Based On Prior Knowledge
• Adaptive wavelet distillation from neural networks through interpretations
• Improving the trustworthiness of image classification models by utilizing bounding-box annotations
• Deep Learning Based Decision Support for Medicine -- A Case Study on Skin Cancer Diagnosis
• Regularizing Explanations in Bayesian Convolutional Neural Networks
• Deep Relevance Regularization: Interpretable and Robust Tumor Typing of Imaging Mass Spectrometry Data
• SIM-ECG: A Signal Importance Mask-driven ECGClassification System