How benign is benign overfitting?
arXiv:2007.04028
Abstract
We investigate two causes for adversarial vulnerability in deep neural networks: bad data and (poorly) trained models. When trained with SGD, deep neural networks essentially achieve zero training error, even in the presence of label noise, while also exhibiting good generalization on natural test data, something referred to as benign overfitting [2, 10]. However, these models are vulnerable to adversarial attacks. We identify label noise as one of the causes for adversarial vulnerability, and provide theoretical and empirical evidence in support of this. Surprisingly, we find several instances of label noise in datasets such as MNIST and CIFAR, and that robustly trained models incur training error on some of these, i.e. they don't fit the noise. However, removing noisy labels alone does not suffice to achieve adversarial robustness. Standard training procedures bias neural networks towards learning "simple" classification boundaries, which may be less robust than more complex ones. We observe that adversarial training does produce more complex decision boundaries. We conjecture that in part the need for complex decision boundaries arises from sub-optimal representation learning. By means of simple toy examples, we show theoretically how the choice of representation can drastically affect adversarial robustness.
References in corpus (18)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Explaining and Harnessing Adversarial Examples
- Shortcut Learning in Deep Neural Networks
- Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
- Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
- Adversarially Robust Generalization Requires More Data
- Fast is better than free: Revisiting adversarial training
- Using Pre-Training Can Improve Model Robustness and Uncertainty
- Natural Adversarial Examples
- Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
- Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate
- Adversarial Training Can Hurt Generalization
- Learning with Bad Training Data via Iterative Trimmed Loss Minimization
- A Closer Look at Accuracy vs. Robustness
- Adversarial Robustness May Be at Odds With Simplicity
- Does data interpolation contradict statistical optimality?
- Complexity of Linear Regions in Deep Networks
- Robustness via Deep Low-Rank Representations