Intriguing properties of adversarial training at scale

Abstract

Adversarial training is one of the main defenses against adversarial attacks. In this paper, we provide the first rigorous study on diagnosing elements of adversarial training, which reveals two intriguing properties. First, we study the role of normalization. Batch normalization (BN) is a crucial element for achieving state-of-the-art performance on many vision tasks, but we show it may prevent networks from obtaining strong robustness in adversarial training. One unexpected observation is that, for models trained with BN, simply removing clean images from training data largely boosts adversarial robustness, i.e., 18.3%. We relate this phenomenon to the hypothesis that clean images and adversarial images are drawn from two different domains. This two-domain hypothesis may explain the issue of BN when training with a mixture of clean and adversarial images, as estimating normalization statistics of this mixture distribution is challenging. Guided by this two-domain hypothesis, we show disentangling the mixture distribution for normalization, i.e., applying separate BNs to clean and adversarial images for statistics estimation, achieves much stronger robustness. Additionally, we find that enforcing BNs to behave consistently at training and testing can further enhance robustness. Second, we study the role of network capacity. We find our so-called "deep" networks are still shallow for the task of adversarial learning. Unlike traditional classification tasks where accuracy is only marginally improved by adding more layers to "deep" networks (e.g., ResNet-152), adversarial training exhibits a much stronger demand on deeper networks to achieve higher adversarial robustness. This robustness improvement can be observed substantially and consistently even by pushing the network capacity to an unprecedented scale, i.e., ResNet-638.

To appear in ICLR 2020

References in corpus (15)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Theoretically Principled Trade-off between Robustness and Accuracy
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Examples Are Not Bugs, They Are Features
• On Detecting Adversarial Perturbations
• Adversarial Robustness through Local Linearization
• Are Labels Required for Improving Adversarial Robustness?
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Adversarial Robustness May Be at Odds With Simplicity
• You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
• Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets
• Convergence of Adversarial Training in Overparametrized Neural Networks
• Adversarial Examples Improve Image Recognition
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Defending Adversarial Attacks by Correcting logits

Cited by in corpus (10)

• The Pitfalls of Simplicity Bias in Neural Networks
• Robust Pre-Training by Adversarial Contrastive Learning
• Convergence of Adversarial Training in Overparametrized Neural Networks
• HYDRA: Pruning Adversarially Robust Neural Networks
• Generative Adversarial Networks: A Survey Towards Private and Secure Applications
• When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks
• Learnable Boundary Guided Adversarial Training
• SoK: Machine Learning Governance
• Robust Sensible Adversarial Learning of Deep Neural Networks for Image Classification
• Domain Invariant Adversarial Learning