Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition

Abstract

Adversarially trained models exhibit a large generalization gap: they can interpolate the training set even for large perturbation radii, but at the cost of large test error on clean samples. To investigate this gap, we decompose the test risk into its bias and variance components and study their behavior as a function of adversarial training perturbation radii ($\epsilon$). We find that the bias increases monotonically with $\epsilon$ and is the dominant term in the risk. Meanwhile, the variance is unimodal as a function of $\epsilon$, peaking near the interpolation threshold for the training set. This characteristic behavior occurs robustly across different datasets and also for other robust training procedures such as randomized smoothing. It thus provides a test for proposed explanations of the generalization gap. We find that some existing explanations fail this test--for instance, by predicting a monotonically increasing variance curve. This underscores the power of bias-variance decompositions in modern settings-by providing two measurements instead of one, they can rule out more explanations than test accuracy alone. We also show that bias and variance can provide useful guidance for scalably reducing the generalization gap, highlighting pre-training and unlabeled data as promising routes.

arXiv commentV2 adds new results and improves organization and presentation

References in corpus (45)

• Adam: A Method for Stochastic Optimization
• Wide Residual Networks
• Reconciling modern machine learning practice and the bias-variance trade-off
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Theoretically Principled Trade-off between Robustness and Accuracy
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Robustness May Be at Odds with Accuracy
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Robustness and Regularization of Support Vector Machines
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Are Labels Required for Improving Adversarial Robustness?
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Adversarial examples from computational constraints
• Do CIFAR-10 Classifiers Generalize to CIFAR-10?
• Rademacher Complexity for Adversarially Robust Generalization
• RobustBench: a standardized adversarial robustness benchmark
• Analyzing the Robustness of Nearest Neighbors to Adversarial Examples
• The Pitfalls of Simplicity Bias in Neural Networks
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• A Closer Look at Accuracy vs. Robustness
• Rethinking Bias-Variance Trade-off for Generalization of Neural Networks
• Testing Robustness Against Unforeseen Adversaries
• Adversarial Robustness May Be at Odds With Simplicity
• Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
• Overfitting in adversarially robust deep learning
• VC Classes are Adversarially Robustly Learnable, but Only Improperly
• Robustness to Adversarial Perturbations in Learning from Incomplete Data
• Lower Bounds on Adversarial Robustness from Optimal Transport
• Adversarial Risk Bounds via Function Transformation
• Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition
• What causes the test error? Going beyond bias-variance via ANOVA
• On the effectiveness of adversarial training against common corruptions
• PAC-learning in the presence of evasion adversaries
• Adversarial Examples from Cryptographic Pseudo-Random Generators
• Kernel regression in high dimensions: Refined analysis beyond double descent
• Inductive Bias of Gradient Descent based Adversarial Training on Separable Data
• Adversarially Robust Estimate and Risk Analysis in Linear Regression
• Asymptotic Behavior of Adversarial Training in Binary Classification
• Vulnerability Under Adversarial Machine Learning: Bias or Variance?

Cited by in corpus (2)

• RobustBench: a standardized adversarial robustness benchmark
• Towards Understanding Generalization via Decomposing Excess Risk Dynamics