Visualizing Deep Networks by Optimizing with Integrated Gradients
arXiv:1905.00954 · doi:10.1609/aaai.v34i07.6863
Abstract
Understanding and interpreting the decisions made by deep learning models is valuable in many domains. In computer vision, computing heatmaps from a deep network is a popular approach for visualizing and understanding deep networks. However, heatmaps that do not correlate with the network may mislead human, hence the performance of heatmaps in providing a faithful explanation to the underlying deep network is crucial. In this paper, we propose I-GOS, which optimizes for a heatmap so that the classification scores on the masked image would maximally decrease. The main novelty of the approach is to compute descent directions based on the integrated gradients instead of the normal gradient, which avoids local optima and speeds up convergence. Compared with previous approaches, our method can flexibly compute heatmaps at any resolution for different user needs. Extensive experiments on several benchmark datasets show that the heatmaps produced by our approach are more correlated with the decision of the underlying deep network, in comparison with other state-of-the-art approaches.
References in corpus (10)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- A Unified Approach to Interpreting Model Predictions
- Striving for Simplicity: The All Convolutional Net
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
- Towards Robust Interpretability with Self-Explaining Neural Networks
- RISE: Randomized Input Sampling for Explanation of Black-box Models
- Real Time Image Saliency for Black Box Classifiers
- A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
- Streaming Weak Submodularity: Interpreting Neural Networks on the Fly