Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?
arXiv:1910.11585
Abstract
Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization. The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study, we explore the mechanisms by which adversarial training improves classifier robustness, and show that these mechanisms can be effectively mimicked using simple regularization methods, including label smoothing and logit squeezing. Remarkably, using these simple regularization methods in combination with Gaussian noise injection, we are able to achieve strong adversarial robustness -- often exceeding that of adversarial training -- using no adversarial examples.
References in corpus (6)
- Ensemble Adversarial Training: Attacks and Defenses
- Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Robust Physical-World Attacks on Deep Learning Models
- Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
Cited by in corpus (10)
- RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
- Bag of Tricks for Adversarial Training
- Adversarial Robustness via Label-Smoothing
- Adversarial Robustness under Long-Tailed Distribution
- An Investigation of how Label Smoothing Affects Generalization
- Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks
- Towards Deep Learning Models Resistant to Large Perturbations
- Understanding the Logit Distributions of Adversarially-Trained Deep Neural Networks
- Constraining Logits by Bounded Function for Adversarial Robustness
- Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness