Robustness of Visual Explanations to Common Data Augmentation
arXiv:2304.08984 · doi:10.1109/CVPRW59228.2023.00381
Abstract
As the use of deep neural networks continues to grow, understanding their behaviour has become more crucial than ever. Post-hoc explainability methods are a potential solution, but their reliability is being called into question. Our research investigates the response of post-hoc visual explanations to naturally occurring transformations, often referred to as augmentations. We anticipate explanations to be invariant under certain transformations, such as changes to the colour map while responding in an equivariant manner to transformations like translation, object scaling, and rotation. We have found remarkable differences in robustness depending on the type of transformation, with some explainability methods (such as LRP composites and Guided Backprop) being more stable than others. We also explore the role of training with data augmentation. We provide evidence that explanations are typically less robust to augmentation than classification performance, regardless of whether data augmentation is used in training or not.
arXiv commentAccepted to The 2nd Explainable AI for Computer Vision (XAI4CV) Workshop at CVPR 2023
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Explaining and Harnessing Adversarial Examples
- Axiomatic Attribution for Deep Networks
- Striving for Simplicity: The All Convolutional Net
- The Effectiveness of Data Augmentation in Image Classification using Deep Learning
- EfficientNetV2: Smaller Models and Faster Training
- Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition
- Captum: A unified and generic model interpretability library for PyTorch
- The (Un)reliability of saliency methods
- Explanations can be manipulated and geometry is to blame
- Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond
- Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy