The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization

Abstract

Adversarial training has shown its ability in producing models that are robust to perturbations on the input data, but usually at the expense of decrease in the standard accuracy. To mitigate this issue, it is commonly believed that more training data will eventually help such adversarially robust models generalize better on the benign/unperturbed test data. In this paper, however, we challenge this conventional belief and show that more training data can hurt the generalization of adversarially robust models in the classification problems. We first investigate the Gaussian mixture classification with a linear loss and identify three regimes based on the strength of the adversary. In the weak adversary regime, more data improves the generalization of adversarially robust models. In the medium adversary regime, with more training data, the generalization loss exhibits a double descent curve, which implies the existence of an intermediate stage where more training data hurts the generalization. In the strong adversary regime, more data almost immediately causes the generalization error to increase. Then we move to the analysis of a two-dimensional classification problem with a 0-1 loss. We prove that more data always hurts the generalization performance of adversarially trained models with large perturbations. To complement our theoretical results, we conduct empirical studies on Gaussian mixture classification, support vector machines (SVMs), and linear regression.

Added theoretical analysis of the Manhattan model and further empirical results

References in corpus (24)

• Reconciling modern machine learning practice and the bias-variance trade-off
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Theoretically Principled Trade-off between Robustness and Accuracy
• Adversarially Robust Generalization Requires More Data
• Adversarial Examples Are Not Bugs, They Are Features
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Learning with a Strong Adversary
• The generalization error of random features regression: Precise asymptotics and double descent curve
• Adversarial Training Can Hurt Generalization
• Adversarial examples from computational constraints
• Rademacher Complexity for Adversarially Robust Generalization
• Image Synthesis with a Single (Robust) Classifier
• Improving the Generalization of Adversarial Training with Domain Adaptation
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Convergence of Adversarial Training in Overparametrized Neural Networks
• VC Classes are Adversarially Robustly Learnable, but Only Improperly
• Lower Bounds on Adversarial Robustness from Optimal Transport
• Generalized No Free Lunch Theorem for Adversarial Robustness
• Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
• Adversarial Risk Bounds via Function Transformation
• More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models
• Lower Bounds for Adversarially Robust PAC Learning
• Efficiently Learning Adversarially Robust Halfspaces with Noise
• A New Perspective for Flexible Feature Gathering in Scene Text Recognition Via Character Anchor Pooling

Cited by in corpus (17)

• Adversarial Training for Large Neural Language Models
• Adversarial Examples Improve Image Recognition
• Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
• Triple descent and the two kinds of overfitting: Where & why do they appear?
• Recent advances in deep learning theory
• Self-supervised Pre-training with Hard Examples Improves Visual Representations
• WeMix: How to Better Utilize Data Augmentation
• Simple Contrastive Representation Adversarial Learning for NLP Tasks
• Provable More Data Hurt in High Dimensional Least Squares Estimator
• Adversarially Robust Low Dimensional Representations
• Asymptotic Behavior of Adversarial Training in Binary Classification
• Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics
• Kodaira-Spencer map, Lagrangian Floer theory and orbifold Jacobian algebras
• ARCH: Efficient Adversarial Regularized Training with Caching
• Noisy Feature Mixup
• Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
• Unique properties of adversarially trained linear classifiers on Gaussian data