Improving the Generalization of Adversarial Training with Domain Adaptation

Abstract

By injecting adversarial examples into training data, adversarial training is promising for improving the robustness of deep learning models. However, most existing adversarial training approaches are based on a specific type of adversarial attack. It may not provide sufficiently representative samples from the adversarial domain, leading to a weak generalization ability on adversarial examples from other attacks. Moreover, during the adversarial training, adversarial perturbations on inputs are usually crafted by fast single-step adversaries so as to scale to large datasets. This work is mainly focused on the adversarial training yet efficient FGSM adversary. In this scenario, it is difficult to train a model with great generalization due to the lack of representative adversarial samples, aka the samples are unable to accurately reflect the adversarial domain. To alleviate this problem, we propose a novel Adversarial Training with Domain Adaptation (ATDA) method. Our intuition is to regard the adversarial training on FGSM adversary as a domain adaption task with limited number of target domain samples. The main idea is to learn a representation that is semantically meaningful and domain invariant on the clean domain as well as the adversarial domain. Empirical evaluations on Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100 demonstrate that ATDA can greatly improve the generalization of adversarial training and the smoothness of the learned models, and outperforms state-of-the-art methods on standard benchmark datasets. To show the transfer ability of our method, we also extend ATDA to the adversarial training on iterative attacks such as PGD-Adversial Training (PAT) and the defense performance is improved considerably.

arXiv commentICLR 2019

References in corpus (9)

• Explaining and Harnessing Adversarial Examples
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Ensemble Adversarial Training: Attacks and Defenses
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• The Space of Transferable Adversarial Examples
• A Closer Look at Memorization in Deep Networks
• Improving Back-Propagation by Adding an Adversarial Gradient

Cited by in corpus (20)

• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks
• Boosting Adversarial Training with Hypersphere Embedding
• Robust Local Features for Improving the Generalization of Adversarial Training
• Natural Language Adversarial Defense through Synonym Encoding
• Adversarial Distributional Training for Robust Deep Learning
• The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization
• Towards Feature Space Adversarial Attack
• Optimal Transport as a Defense Against Adversarial Attacks
• Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability
• Feature Prioritization and Regularization Improve Standard Accuracy and Adversarial Robustness
• Towards Deep Learning Models Resistant to Large Perturbations
• Adversarial Feature Desensitization
• Blind Adversarial Training: Balance Accuracy and Robustness
• On Norm-Agnostic Robustness of Adversarial Training
• Adv-4-Adv: Thwarting Changing Adversarial Perturbations via Adversarial Domain Adaptation
• Data Augmentation via Structured Adversarial Perturbations
• Blind Adversarial Pruning: Balance Accuracy, Efficiency and Robustness
• Domain Invariant Adversarial Learning
• Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification