Restricting the Flow: Information Bottlenecks for Attribution

Abstract

Attribution methods provide insights into the decision-making of machine learning models like artificial neural networks. For a given input sample, they assign a relevance score to each individual input variable, such as the pixels of an image. In this work we adapt the information bottleneck concept for attribution. By adding noise to intermediate feature maps we restrict the flow of information and can quantify (in bits) how much information image regions provide. We compare our method against ten baselines using three different metrics on VGG-16 and ResNet-50, and find that our methods outperform all baselines in five out of six settings. The method's information-theoretic foundation provides an absolute frame of reference for attribution values (bits) and a guarantee that regions scored close to zero are not necessary for the network's decision. For reviews: https://openreview.net/forum?id=S1xWh1rYwB For code: https://github.com/BioroboticsLab/IBA

arXiv comment18 pages, 12 figures, accepted at ICLR 2020 (Oral)

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• The information bottleneck method
• Learning Important Features Through Propagating Activation Differences
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• SmoothGrad: removing noise by adding noise
• How to Explain Individual Classification Decisions
• Sanity Checks for Saliency Maps
• Variational Dropout and the Local Reparameterization Trick
• Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
• Visualizing and Understanding Atari Agents
• On Variational Bounds of Mutual Information

Cited by in corpus (21)

• Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps
• On Interpretability of Artificial Neural Networks: A Survey
• When Explanations Lie: Why Many Modified BP Attributions Fail
• Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking
• Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Prediction
• Smoothed Geometry for Robust Attribution
• The Convex Information Bottleneck Lagrangian
• Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Exploration
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Human-interpretable model explainability on high-dimensional data
• Improving Feature Attribution through Input-specific Network Pruning
• Explainable Abstract Trains Dataset
• Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
• FastSHAP: Real-Time Shapley Value Estimation
• Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoring
• Automatic Neural Network Pruning that Efficiently Preserves the Model Accuracy
• Inserting Information Bottlenecks for Attribution in Transformers
• Mutual Information Preserving Back-propagation: Learn to Invert for Faithful Attribution
• Explaining Representation by Mutual Information
• When Stability meets Sufficiency: Informative Explanations that do not Overwhelm
• Causal Representation Learning for Context-Aware Face Transfer