Overfitting in adversarially robust deep learning

Abstract

It is common practice in deep learning to use overparameterized networks and train for as long as possible; there are numerous studies that show, both theoretically and empirically, that such practices surprisingly do not unduly harm the generalization performance of the classifier. In this paper, we empirically study this phenomenon in the setting of adversarially trained deep networks, which are trained to minimize the loss under worst-case adversarial perturbations. We find that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training across multiple datasets (SVHN, CIFAR-10, CIFAR-100, and ImageNet) and perturbation models (${\ell }_{\infty }$ and ${\ell }_2$). Based upon this observed effect, we show that the performance gains of virtually all recent algorithmic improvements upon adversarial training can be matched by simply using early stopping. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting. Finally, we study several classical and modern deep learning remedies for overfitting, including regularization and data augmentation, and find that no approach in isolation improves significantly upon the gains achieved by early stopping. All code for reproducing the experiments as well as pretrained model weights and training logs can be found at https://github.com/locuslab/robust_overfitting.

References in corpus (31)

• mixup: Beyond Empirical Risk Minimization
• Improved Regularization of Convolutional Neural Networks with Cutout
• Wide Residual Networks
• Reconciling modern machine learning practice and the bias-variance trade-off
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Understanding deep learning requires rethinking generalization
• Theoretically Principled Trade-off between Robustness and Accuracy
• Synthesizing Robust Adversarial Examples
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Fast is better than free: Revisiting adversarial training
• Countering Adversarial Images using Input Transformations
• Detecting Adversarial Samples from Artifacts
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• Certifying Some Distributional Robustness with Principled Adversarial Training
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Spatially Transformed Adversarial Examples
• NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
• Semidefinite relaxations for certifying robustness to adversarial examples
• A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
• Do CIFAR-10 Classifiers Generalize to CIFAR-10?
• Rademacher Complexity for Adversarially Robust Generalization
• Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
• Unlabeled Data Improves Adversarial Robustness
• ME-Net: Towards Effective Adversarial Robustness with Matrix Estimation
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?
• A Continuous-Time View of Early Stopping for Least Squares

Cited by in corpus (21)

• A Closer Look at Accuracy vs. Robustness
• Robust Pre-Training by Adversarial Contrastive Learning
• Boosting Adversarial Training with Hypersphere Embedding
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• Understanding and Improving Fast Adversarial Training
• Learning perturbation sets for robust machine learning
• On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
• Towards Understanding Fast Adversarial Training
• On the Generalization Properties of Adversarial Training
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Context-Aware Safe Reinforcement Learning for Non-Stationary Environments
• Optimizing Mode Connectivity via Neuron Alignment
• Train simultaneously, generalize better: Stability of gradient-based minimax learners
• GAN-based Data Augmentation for Chest X-ray Classification
• Internal Wasserstein Distance for Adversarial Attack and Defense
• RAIN: A Simple Approach for Robust and Accurate Image Classification Networks
• Understanding Catastrophic Overfitting in Adversarial Training
• Improving Model Robustness with Latent Distribution Locally and Globally
• Dynamic Efficient Adversarial Training Guided by Gradient Magnitude
• Adversarial Training with Stochastic Weight Average
• Bridged Adversarial Training