Interpreting Adversarial Examples by Activation Promotion and Suppression

Abstract

It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers. However, interpretability of these perturbations is less explored in the literature. This work aims to better understand the roles of adversarial perturbations and provide visual explanations from pixel, image and network perspectives. We show that adversaries have a promotion-suppression effect (PSE) on neurons' activations and can be primarily categorized into three types: i) suppression-dominated perturbations that mainly reduce the classification score of the true label, ii) promotion-dominated perturbations that focus on boosting the confidence of the target label, and iii) balanced perturbations that play a dual role in suppression and promotion. We also provide image-level interpretability of adversarial examples. This links PSE of pixel-level perturbations to class-specific discriminative image regions localized by class activation mapping (Zhou et al. 2016). Further, we examine the adversarial effect through network dissection (Bau et al. 2017), which offers concept-level interpretability of hidden units. We show that there exists a tight connection between the units' sensitivity to adversarial attacks and their interpretability on semantic concepts. Lastly, we provide some new insights from our interpretation to improve the adversarial robustness of networks.

References in corpus (12)

• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Spatially Transformed Adversarial Examples
• LaVAN: Localized and Visible Adversarial Noise
• Structured Adversarial Attack: Towards General Implementation and Better Interpretability
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Improved robustness to adversarial examples using Lipschitz regularization of the loss
• Interpretable Deep Learning under Fire
• RANDOM MASK: Towards Robust Convolutional Neural Networks
• ML-LOO: Detecting Adversarial Examples with Feature Attribution
• ASP:A Fast Adversarial Attack Example Generation Framework based on Adversarial Saliency Prediction

Cited by in corpus (17)

• Interpreting and Improving Adversarial Robustness of Deep Neural Networks with Neuron Sensitivity
• Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• On Interpretability of Artificial Neural Networks: A Survey
• NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
• Adversarial T-shirt! Evading Person Detectors in A Physical World
• Defending against Backdoor Attack on Deep Neural Networks
• On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Method
• CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection
• Garment Design with Generative Adversarial Networks
• Enhancing Intrinsic Adversarial Robustness via Feature Pyramid Decoder
• Patch-wise Attack for Fooling Deep Neural Network
• A Unified Game-Theoretic Interpretation of Adversarial Robustness
• Understanding and Diagnosing Vulnerability under Adversarial Attacks
• Heat and Blur: An Effective and Fast Defense Against Adversarial Examples
• Towards an Efficient and General Framework of Robust Training for Graph Neural Networks
• Visualizing Representations of Adversarially Perturbed Inputs