Learnable Boundary Guided Adversarial Training

Abstract

Previous adversarial training raises model robustness under the compromise of accuracy on natural data. In this paper, we reduce natural accuracy degradation. We use the model logits from one clean model to guide learning of another one robust model, taking into consideration that logits from the well trained clean model embed the most discriminative features of natural data, {\it e.g.}, generalizable classifier boundary. Our solution is to constrain logits from the robust model that takes adversarial examples as input and makes it similar to those from the clean model fed with corresponding natural data. It lets the robust model inherit the classifier boundary of the clean model. Moreover, we observe such boundary guidance can not only preserve high natural accuracy but also benefit model robustness, which gives new insights and facilitates progress for the adversarial community. Finally, extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet testify to the effectiveness of our method. We achieve new state-of-the-art robustness on CIFAR-100 without additional real or synthetic data with auto-attack benchmark \footnote{\url{https://github.com/fra31/auto-attack}}. Our code is available at \url{https://github.com/dvlab-research/LBGAT}.

arXiv commentICCV2021

References in corpus (13)

• Distilling the Knowledge in a Neural Network
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• DARTS: Differentiable Architecture Search
• Theoretically Principled Trade-off between Robustness and Accuracy
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Fast is better than free: Revisiting adversarial training
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Data-Free Knowledge Distillation for Deep Neural Networks
• Adversarial Robustness through Local Linearization
• Deep Model Compression: Distilling Knowledge from Noisy Teachers
• Metric Learning for Adversarial Robustness
• Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models
• Intriguing properties of adversarial training at scale

Cited by in corpus (7)

• Fixing Data Augmentation to Improve Adversarial Robustness
• Attacking Adversarial Attacks as A Defense
• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training
• Adversarial Visual Robustness by Causal Intervention
• Improving Robustness using Generated Data
• Data Quality Matters For Adversarial Training: An Empirical Study
• Dual Head Adversarial Training