Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples

Abstract

Adversarial training and its variants have become de facto standards for learning robust deep neural networks. In this paper, we explore the landscape around adversarial training in a bid to uncover its limits. We systematically study the effect of different training losses, model sizes, activation functions, the addition of unlabeled data (through pseudo-labeling) and other factors on adversarial robustness. We discover that it is possible to train robust models that go well beyond state-of-the-art results by combining larger models, Swish/SiLU activations and model weight averaging. We demonstrate large improvements on CIFAR-10 and CIFAR-100 against ${\ell }_{\infty }$ and ${\ell }_2$ norm-bounded perturbations of size $8/255$ and $128/255$, respectively. In the setting with additional unlabeled data, we obtain an accuracy under attack of 65.88% against ${\ell }_{\infty }$ perturbations of size $8/255$ on CIFAR-10 (+6.35% with respect to prior art). Without additional data, we obtain an accuracy under attack of 57.20% (+3.46%). To test the generality of our findings and without any additional modifications, we obtain an accuracy under attack of 80.53% (+7.62%) against ${\ell }_2$ perturbations of size $128/255$ on CIFAR-10, and of 36.88% (+8.46%) against ${\ell }_{\infty }$ perturbations of size $8/255$ on CIFAR-100. All models are available at https://github.com/deepmind/deepmind-research/tree/master/adversarial_robustness.

Fixed minor formatting issues and added link to models

References in corpus (34)

• Gaussian Error Linear Units (GELUs)
• End to End Learning for Self-Driving Cars
• Improved Regularization of Convolutional Neural Networks with Cutout
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Ensemble Adversarial Training: Attacks and Defenses
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Searching for Activation Functions
• AutoAugment: Learning Augmentation Policies from Data
• Certified Adversarial Robustness via Randomized Smoothing
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• On Evaluating Adversarial Robustness
• Adversarially Robust Generalization Requires More Data
• Fast is better than free: Revisiting adversarial training
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Detecting Adversarial Samples from Artifacts
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Defensive Distillation is Not Robust to Adversarial Examples
• On Detecting Adversarial Perturbations
• NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• Adversarial Robustness through Local Linearization
• Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
• Smooth Adversarial Training
• Self-Adaptive Training: beyond Empirical Risk Minimization
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing
• Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks
• Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets
• An Alternative Surrogate Loss for PGD-based Adversarial Testing
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?

Cited by in corpus (50)

• How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses
• Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling
• RobustBench: a standardized adversarial robustness benchmark
• Fixing Data Augmentation to Improve Adversarial Robustness
• A Review of Adversarial Attack and Defense for Classification Methods
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Adversarial purification with Score-based generative models
• Geometry-aware Instance-reweighted Adversarial Training
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
• What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
• SAT: Improving Adversarial Training via Curriculum-Based Loss Smoothing
• On the effectiveness of adversarial training against common corruptions
• Exploring Memorization in Adversarial Training
• Attacking Adversarial Attacks as A Defense
• Bag of Tricks for Adversarial Training
• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training
• Post-breach Recovery: Protection against White-box Adversarial Examples for Leaked DNN Models
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Adversarial Robustness under Long-Tailed Distribution
• Adversarial Token Attacks on Vision Transformers
• Dense Hopfield Networks in the Teacher-Student Setting
• Random Noise Defense Against Query-Based Black-Box Attacks
• Adversarial Robustness through the Lens of Convolutional Filters
• Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?
• DiffDefense: Defending against Adversarial Attacks via Diffusion Models
• Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
• Mind the box: $l_1$-APGD for sparse adversarial attacks on image classifiers
• A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness
• Improving Robustness using Generated Data
• AccelAT: A Framework for Accelerating the Adversarial Training of Deep Neural Networks through Accuracy Gradient
• Automated Discovery of Adaptive Attacks on Adversarial Defenses
• Data Quality Matters For Adversarial Training: An Empirical Study
• Defending Against Image Corruptions Through Adversarial Augmentations
• LAFEAT: Piercing Through Adversarial Defenses with Latent Features
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• Improving Model Robustness with Latent Distribution Locally and Globally
• Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks
• Mixed Nash Equilibria in the Adversarial Examples Game
• Adversarial Feature Desensitization
• Identifying Untrustworthy Predictions in Neural Networks by Geometric Gradient Analysis
• Adversarial Attacks on ML Defense Models Competition
• Lower Bounds on Cross-Entropy Loss in the Presence of Test-time Adversaries
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness
• Training Provably Robust Models by Polyhedral Envelope Regularization
• Bridged Adversarial Training
• Robustifying ${\ell }_{\infty }$ Adversarial Training to the Union of Perturbation Models
• Closing the Gap: Achieving Better Accuracy-Robustness Tradeoffs against Query-Based Attacks
• Shape Defense Against Adversarial Attacks
• Training Efficiency and Robustness in Deep Learning
• Less is More: Feature Selection for Adversarial Robustness with Compressive Counter-Adversarial Attacks