Improving Feature Attribution through Input-specific Network Pruning
arXiv:1911.11081
Abstract
Attributing the output of a neural network to the contribution of given input elements is a way of shedding light on the black-box nature of neural networks. Due to the complexity of current network architectures, current gradient-based attribution methods provide very noisy or coarse results. We propose to prune a neural network for a given single input to keep only neurons that highly contribute to the prediction. We show that by input-specific pruning, network gradients change from reflecting local (noisy) importance information to global importance. Our proposed method is efficient and generates fine-grained attribution maps. We further provide a theoretical justification of the pruning approach relating it to perturbations and validate it through a novel experimental setup. Our method is evaluated by multiple benchmarks: sanity checks, pixel perturbation, and Remove-and-Retrain (ROAR). These benchmarks evaluate the method from different perspectives and our method performs better than other methods across all evaluations.
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Axiomatic Attribution for Deep Networks
- Learning Important Features Through Propagating Activation Differences
- Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
- How to Explain Individual Classification Decisions
- Sanity Checks for Saliency Maps
- A Benchmark for Interpretability Methods in Deep Neural Networks
- RISE: Randomized Input Sampling for Explanation of Black-box Models
- PerforatedCNNs: Acceleration through Elimination of Redundant Convolutions
- Investigating the influence of noise and distractors on the interpretation of neural networks
- Full-Gradient Representation for Neural Network Visualization
- Restricting the Flow: Information Bottlenecks for Attribution