Efficient Adversarial Training with Transferable Adversarial Examples

Abstract

Adversarial training is an effective defense method to protect classification models against adversarial attacks. However, one limitation of this approach is that it can require orders of magnitude additional training time due to high cost of generating strong adversarial examples during training. In this paper, we first show that there is high transferability between models from neighboring epochs in the same training process, i.e., adversarial examples from one epoch continue to be adversarial in subsequent epochs. Leveraging this property, we propose a novel method, Adversarial Training with Transferable Adversarial Examples (ATTA), that can enhance the robustness of trained models and greatly improve the training efficiency by accumulating adversarial perturbations through epochs. Compared to state-of-the-art adversarial training methods, ATTA enhances adversarial accuracy by up to 7.2% on CIFAR10 and requires 12~14x less training time on MNIST and CIFAR10 datasets with comparable model robustness.

arXiv commentProceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2020 (CVPR 2020)

References in corpus (17)

• Ensemble Adversarial Training: Attacks and Defenses
• Wide Residual Networks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Robust Physical-World Attacks on Deep Learning Models
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Examples Are Not Bugs, They Are Features
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN
• Improving Transferability of Adversarial Examples with Input Diversity
• Unlabeled Data Improves Adversarial Robustness
• The Limitations of Adversarial Training and the Blind-Spot Attack
• Curriculum Adversarial Training

Cited by in corpus (3)

• Attacking and Defending Machine Learning Applications of Public Cloud
• Understanding and Diagnosing Vulnerability under Adversarial Attacks
• Do Deep Minds Think Alike? Selective Adversarial Attacks for Fine-Grained Manipulation of Multiple Deep Neural Networks