Improving Adversarial Robustness via Channel-wise Activation Suppressing

Abstract

The study of adversarial examples and their activation has attracted significant attention for secure and robust learning with deep neural networks (DNNs). Different from existing works, in this paper, we highlight two new characteristics of adversarial examples from the channel-wise activation perspective: 1) the activation magnitudes of adversarial examples are higher than that of natural examples; and 2) the channels are activated more uniformly by adversarial examples than natural examples. We find that the state-of-the-art defense adversarial training has addressed the first issue of high activation magnitudes via training on adversarial examples, while the second issue of uniform activation remains. This motivates us to suppress redundant activation from being activated by adversarial perturbations via a Channel-wise Activation Suppressing (CAS) strategy. We show that CAS can train a model that inherently suppresses adversarial activation, and can be easily applied to existing defense methods to further improve their robustness. Our work provides a simple but generic training strategy for robustifying the intermediate layer activation of DNNs.

arXiv commentICLR2021 accepted paper

References in corpus (16)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Efficient Neural Network Robustness Certification with General Activation Functions
• Stochastic Activation Pruning for Robust Adversarial Defense
• Residual Convolutional CTC Networks for Automatic Speech Recognition
• Batch Normalization is a Cause of Adversarial Vulnerability
• Overfitting in adversarially robust deep learning
• NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
• Interpreting Adversarial Examples by Activation Promotion and Suppression
• Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions
• Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models
• Improved robustness to adversarial examples using Lipschitz regularization of the loss
• Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization

Cited by in corpus (19)

• Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuning
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
• Reliable Adversarial Distillation with Unreliable Teachers
• Understanding the Interaction of Adversarial Training with Noisy Labels
• CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection
• Adversarial Robustness through the Lens of Convolutional Filters
• Instance Correction for Learning with Open-set Noisy Labels
• Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness
• Guided Interpolation for Adversarial Training
• Towards an Awareness of Time Series Anomaly Detection Models' Adversarial Vulnerability
• Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
• Learning Diverse-Structured Networks for Adversarial Robustness
• Analysis and Applications of Class-wise Robustness in Adversarial Training
• Adversarial Feature Desensitization
• Clustering Effect of (Linearized) Adversarial Robust Models
• A Unified Game-Theoretic Interpretation of Adversarial Robustness
• Less is More: Feature Selection for Adversarial Robustness with Compressive Counter-Adversarial Attacks
• Identifying Layers Susceptible to Adversarial Attacks
• Dual Head Adversarial Training