Exploring Memorization in Adversarial Training
arXiv:2106.01606
Abstract
Deep learning models have a propensity for fitting the entire training set even with random labels, which requires memorization of every training sample. In this paper, we explore the memorization effect in adversarial training (AT) for promoting a deeper understanding of model capacity, convergence, generalization, and especially robust overfitting of the adversarially trained models. We first demonstrate that deep networks have sufficient capacity to memorize adversarial examples of training data with completely random labels, but not all AT algorithms can converge under the extreme circumstance. Our study of AT with random labels motivates further analyses on the convergence and generalization of AT. We find that some AT approaches suffer from a gradient instability issue and most recently suggested complexity measures cannot explain robust generalization by considering models trained on random labels. Furthermore, we identify a significant drawback of memorization in AT that it could result in robust overfitting. We then propose a new mitigation algorithm motivated by detailed memorization analyses. Extensive experiments on various datasets validate the effectiveness of the proposed method.
arXiv commentAccepted by ICLR 2022. 24 pages
References in corpus (25)
- Reconciling modern machine learning practice and the bias-variance trade-off
- Theoretically Principled Trade-off between Robustness and Accuracy
- Learning to Reweight Examples for Robust Deep Learning
- A Convergence Theory for Deep Learning via Over-Parameterization
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Certified Adversarial Robustness via Randomized Smoothing
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
- Adversarially Robust Generalization Requires More Data
- Fast is better than free: Revisiting adversarial training
- Using Pre-Training Can Improve Model Robustness and Uncertainty
- A Closer Look at Memorization in Deep Networks
- Sensitivity and Generalization in Neural Networks: an Empirical Study
- Adversarial Robustness through Local Linearization
- Improving Adversarial Robustness via Promoting Ensemble Diversity
- On the Convergence and Robustness of Adversarial Training
- Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
- Adversarial examples from computational constraints
- Rademacher Complexity for Adversarially Robust Generalization
- Adversarially Robust Generalization Just Requires More Unlabeled Data
- Norm-Based Capacity Control in Neural Networks
- Metric Learning for Adversarial Robustness
- Convergence of Adversarial Training in Overparametrized Neural Networks
- On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
- Adversarial Distributional Training for Robust Deep Learning
- Theoretical Analysis of Adversarial Learning: A Minimax Approach