On the Connection Between Adversarial Robustness and Saliency Map Interpretability

Abstract

Recent studies on the adversarial vulnerability of neural networks have shown that models trained to be more robust to adversarial attacks exhibit more interpretable saliency maps than their non-robust counterparts. We aim to quantify this behavior by considering the alignment between input image and saliency map. We hypothesize that as the distance to the decision boundary grows,so does the alignment. This connection is strictly true in the case of linear models. We confirm these theoretical findings with experiments based on models trained with a local Lipschitz regularization and identify where the non-linear nature of neural networks weakens the relation.

12 pages, accepted for publication at the 36th International Conference on Machine Learning 2019

Cited by in corpus (15)

• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free
• Quantifying the Preferential Direction of the Model Gradient in Adversarial Training With Projected Gradient Descent
• Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization
• Second Order Optimization for Adversarial Robustness and Interpretability
• Robust Models Are More Interpretable Because Attributions Look Normal
• Adversarially Robust Estimate and Risk Analysis in Linear Regression
• Understanding and Diagnosing Vulnerability under Adversarial Attacks
• Identifying Untrustworthy Predictions in Neural Networks by Geometric Gradient Analysis
• An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
• Noise Modulation: Let Your Model Interpret Itself
• Leave-one-out Unfairness
• Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approach