Towards Understanding Fast Adversarial Training

Abstract

Current neural-network-based classifiers are susceptible to adversarial examples. The most empirically successful approach to defending against such adversarial examples is adversarial training, which incorporates a strong self-attack during training to enhance its robustness. This approach, however, is computationally expensive and hence is hard to scale up. A recent work, called fast adversarial training, has shown that it is possible to markedly reduce computation time without sacrificing significant performance. This approach incorporates simple self-attacks, yet it can only run for a limited number of training epochs, resulting in sub-optimal performance. In this paper, we conduct experiments to understand the behavior of fast adversarial training and show the key to its success is the ability to recover from overfitting to weak attacks. We then extend our findings to improve fast adversarial training, demonstrating superior robust accuracy to strong adversarial training, with much-reduced training time.

References in corpus (16)

• Wide Residual Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Mixed Precision Training
• Theoretically Principled Trade-off between Robustness and Accuracy
• On Evaluating Adversarial Robustness
• Certified Defenses against Adversarial Examples
• Fast is better than free: Revisiting adversarial training
• Certified Adversarial Robustness with Additive Noise
• On the Convergence and Robustness of Adversarial Training
• A Dual Approach to Scalable Verification of Deep Networks
• Towards Fast Computation of Certified Robustness for ReLU Networks
• On Adaptive Attacks to Adversarial Example Defenses
• Scaling provable adversarial defenses
• Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
• Overfitting in adversarially robust deep learning

Cited by in corpus (10)

• Recent Advances in Adversarial Training for Adversarial Robustness
• Adversarial purification with Score-based generative models
• A Real-time Defense against Website Fingerprinting Attacks
• Bag of Tricks for Adversarial Training
• Understanding Catastrophic Overfitting in Adversarial Training
• ZeroGrad : Mitigating and Explaining Catastrophic Overfitting in FGSM Adversarial Training
• Multi-objective Search of Robust Neural Architectures against Multiple Types of Adversarial Attacks
• Multi-stage Optimization based Adversarial Training
• Style Curriculum Learning for Robust Medical Image Segmentation
• Robust Single-step Adversarial Training with Regularizer