Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis
arXiv:2111.04138
Abstract
We describe a novel attribution method which is grounded in Sensitivity Analysis and uses Sobol indices. Beyond modeling the individual contributions of image regions, Sobol indices provide an efficient way to capture higher-order interactions between image regions and their contributions to a neural network's prediction through the lens of variance. We describe an approach that makes the computation of these indices efficient for high-dimensional problems by using perturbation masks coupled with efficient estimators to handle the high dimensionality of images. Importantly, we show that the proposed method leads to favorable scores on standard benchmarks for vision (and language models) while drastically reducing the computing time compared to other black-box methods -- even surpassing the accuracy of state-of-the-art white-box methods which require access to internal representations. Our code is freely available: https://github.com/fel-thomas/Sobol-Attribution-Method
arXiv commentNeurIPS2021
References in corpus (11)
- Axiomatic Attribution for Deep Networks
- Learning Important Features Through Propagating Activation Differences
- SmoothGrad: removing noise by adding noise
- How to Explain Individual Classification Decisions
- Sanity Checks for Saliency Maps
- Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
- A Benchmark for Interpretability Methods in Deep Neural Networks
- "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach
- Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering
- A comprehensive comparison of total-order estimators for global sensitivity analysis
- Learning what and where to attend