Deeply Explain CNN via Hierarchical Decomposition
arXiv:2201.09205 · doi:10.1007/s11263-022-01746-x
Abstract
In computer vision, some attribution methods for explaining CNNs attempt to study how the intermediate features affect the network prediction. However, they usually ignore the feature hierarchies among the intermediate features. This paper introduces a hierarchical decomposition framework to explain CNN's decision-making process in a top-down manner. Specifically, we propose a gradient-based activation propagation (gAP) module that can decompose any intermediate CNN decision to its lower layers and find the supporting features. Then we utilize the gAP module to iteratively decompose the network decision to the supporting evidence from different CNN layers. The proposed framework can generate a deep hierarchy of strongly associated supporting evidence for the network decision, which provides insight into the decision-making process. Moreover, gAP is effort-free for understanding CNN-based models without network architecture modification and extra training process. Experiments show the effectiveness of the proposed method. The code and interactive demo website will be made publicly available.
References in corpus (22)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- A Survey on Deep Learning in Medical Image Analysis
- Explaining and Harnessing Adversarial Examples
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
- Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
- Learning Important Features Through Propagating Activation Differences
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Understanding Neural Networks Through Deep Visualization
- Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
- Interpretable Explanations of Black Boxes by Meaningful Perturbation
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- How to Explain Individual Classification Decisions
- This Looks Like That: Deep Learning for Interpretable Image Recognition
- Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
- Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
- Understanding the Role of Individual Units in a Deep Neural Network
- Learning how to explain neural networks: PatternNet and PatternAttribution
- RISE: Randomized Input Sampling for Explanation of Black-box Models
- Real Time Image Saliency for Black Box Classifiers
- Full-Gradient Representation for Neural Network Visualization
- Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs