Robust Local Features for Improving the Generalization of Adversarial Training

Abstract

Adversarial training has been demonstrated as one of the most effective methods for training robust models to defend against adversarial examples. However, adversarially trained models often lack adversarially robust generalization on unseen testing data. Recent works show that adversarially trained models are more biased towards global structure features. Instead, in this work, we would like to investigate the relationship between the generalization of adversarial training and the robust local features, as the robust local features generalize well for unseen shape variation. To learn the robust local features, we develop a Random Block Shuffle (RBS) transformation to break up the global structure features on normal adversarial examples. We continue to propose a new approach called Robust Local Features for Adversarial Training (RLFAT), which first learns the robust local features by adversarial training on the RBS-transformed adversarial examples, and then transfers the robust local features into the training of normal adversarial examples. To demonstrate the generality of our argument, we implement RLFAT in currently state-of-the-art adversarial training frameworks. Extensive experiments on STL-10, CIFAR-10 and CIFAR-100 show that RLFAT significantly improves both the adversarially robust generalization and the standard generalization of adversarial training. Additionally, we demonstrate that our models capture more local features of the object on the images, aligning better with human perception.

accepted by ICLR 2020

References in corpus (12)

• Theoretically Principled Trade-off between Robustness and Accuracy
• Delving into Transferable Adversarial Examples and Black-box Attacks
• SmoothGrad: removing noise by adding noise
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Adversarially Robust Generalization Requires More Data
• Adversarial Examples Are Not Bugs, They Are Features
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Improving the Generalization of Adversarial Training with Domain Adaptation
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Interpreting Adversarially Trained Convolutional Neural Networks
• The Limitations of Adversarial Training and the Blind-Spot Attack
• A Direct Approach to Robust Deep Learning Using Adversarial Networks

Cited by in corpus (7)

• Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks
• Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability
• Weight-Covariance Alignment for Adversarially Robust Neural Networks
• Adversarial Feature Desensitization
• Meta Gradient Adversarial Attack
• AN-GCN: An Anonymous Graph Convolutional Network Defense Against Edge-Perturbing Attack
• Adaptive Perturbation for Adversarial Attack