A simple defense against adversarial attacks on heatmap explanations
arXiv:2007.06381
Abstract
With machine learning models being used for more sensitive applications, we rely on interpretability methods to prove that no discriminating attributes were used for classification. A potential concern is the so-called "fair-washing" - manipulating a model such that the features used in reality are hidden and more innocuous features are shown to be important instead. In our work we present an effective defence against such adversarial attacks on neural networks. By a simple aggregation of multiple explanation methods, the network becomes robust against manipulation. This holds even when the attacker has exact knowledge of the model weights and the explanation methods used.
Accepted at 2020 Workshop on Human Interpretability in Machine Learning (WHI)
References in corpus (10)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Axiomatic Attribution for Deep Networks
- Learning Important Features Through Propagating Activation Differences
- Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
- SmoothGrad: removing noise by adding noise
- Sanity Checks for Saliency Maps
- Explanations can be manipulated and geometry is to blame
- A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
- Fairwashing: the risk of rationalization
- Interpretable Deep Learning under Fire