Information-Theoretic Visual Explanation for Black-Box Classifiers
arXiv:2009.11150
Abstract
In this work, we attempt to explain the prediction of any black-box classifier from an information-theoretic perspective. For each input feature, we compare the classifier outputs with and without that feature using two information-theoretic metrics. Accordingly, we obtain two attribution maps--an information gain (IG) map and a point-wise mutual information (PMI) map. IG map provides a class-independent answer to "How informative is each pixel?", and PMI map offers a class-specific explanation of "How much does each pixel support a specific class?" Compared to existing methods, our method improves the correctness of the attribution maps in terms of a quantitative metric. We also provide a detailed analysis of an ImageNet classifier using the proposed method, and the code is available online.
References in corpus (13)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Axiomatic Attribution for Deep Networks
- Striving for Simplicity: The All Convolutional Net
- Learning Important Features Through Propagating Activation Differences
- SmoothGrad: removing noise by adding noise
- Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
- Sanity Checks for Saliency Maps
- Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
- A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
- HexaGAN: Generative Adversarial Nets for Real World Classification
- Interpretation of NLP models through input marginalization