Understanding Catastrophic Overfitting in Adversarial Training

Abstract

Recently, FGSM adversarial training is found to be able to train a robust model which is comparable to the one trained by PGD but an order of magnitude faster. However, there is a failure mode called catastrophic overfitting (CO) that the classifier loses its robustness suddenly during the training and hardly recovers by itself. In this paper, we find CO is not only limited to FGSM, but also happens in $\mbox{DF}^{\infty}$-1 adversarial training. Then, we analyze the geometric properties for both FGSM and $\mbox{DF}^{\infty}$-1 and find they have totally different decision boundaries after CO. For FGSM, a new decision boundary is generated along the direction of perturbation and makes the small perturbation more effective than the large one. While for $\mbox{DF}^{\infty}$-1, there is no new decision boundary generated along the direction of perturbation, instead the perturbation generated by $\mbox{DF}^{\infty}$-1 becomes smaller after CO and thus loses its effectiveness. We also experimentally analyze three hypotheses on potential factors causing CO. And then based on the empirical analysis, we modify the RS-FGSM by not projecting perturbation back to the $l_{\infty }$ ball. By this small modification, we could achieve $47.56\pm 0.37\%$ PGD-50-10 accuracy on CIFAR10 with $\epsilon =8/255$ in contrast to $43.57\pm 0.30\%$ by RS-FGSM and also further extend the working range of $\epsilon$ from 8/255 to 11/255 on CIFAR10 without CO occurring.

References in corpus (16)

• Intriguing properties of neural networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Certified Defenses against Adversarial Examples
• Fast is better than free: Revisiting adversarial training
• Countering Adversarial Images using Input Transformations
• Adversarial Machine Learning at Scale
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Adversarial Robustness through Local Linearization
• You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
• Overfitting in adversarially robust deep learning
• Understanding and Improving Fast Adversarial Training
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Towards Understanding Fast Adversarial Training