Noise-adding Methods of Saliency Map as Series of Higher Order Partial Derivative
arXiv:1806.03000
Abstract
SmoothGrad and VarGrad are techniques that enhance the empirical quality of standard saliency maps by adding noise to input. However, there were few works that provide a rigorous theoretical interpretation of those methods. We analytically formalize the result of these noise-adding methods. As a result, we observe two interesting results from the existing noise-adding methods. First, SmoothGrad does not make the gradient of the score function smooth. Second, VarGrad is independent of the gradient of the score function. We believe that our findings provide a clue to reveal the relationship between local explanation methods of deep neural networks and higher-order partial derivatives of the score function.
arXiv commentpresented at 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018), Stockholm, Sweden
Cited by in corpus (5)
- Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI
- Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning
- Interpreting Super-Resolution Networks with Local Attribution Maps
- Rationalizing Predictions by Adversarial Information Calibration
- Saliency strikes back: How filtering out high frequencies improves white-box explanations