On the effectiveness of adversarial training against common corruptions

Abstract

The literature on robustness towards common corruptions shows no consensus on whether adversarial training can improve the performance in this setting. First, we show that, when used with an appropriately selected perturbation radius, ${\ell }_p$ adversarial training can serve as a strong baseline against common corruptions improving both accuracy and calibration. Then we explain why adversarial training performs better than data augmentation with simple Gaussian noise which has been observed to be a meaningful baseline on common corruptions. Related to this, we identify the $\sigma$-overfitting phenomenon when Gaussian augmentation overfits to a particular standard deviation used for training which has a significant detrimental effect on common corruption accuracy. We discuss how to alleviate this problem and then how to further enhance ${\ell }_p$ adversarial training by introducing an efficient relaxation of adversarial training with learned perceptual image patch similarity as the distance metric. Through experiments on CIFAR-10 and ImageNet-100, we show that our approach does not only improve the ${\ell }_p$ adversarial training baseline but also has cumulative gains with data augmentation methods such as AugMix, DeepAugment, ANT, and SIN, leading to state-of-the-art performance on common corruptions. The code of our experiments is publicly available at https://github.com/tml-epfl/adv-training-corruptions.

arXiv commentNew calibration results, more comprehensive experimental evaluation (e.g., new results with AugMix+JSD and DeepAugment)

References in corpus (28)

• Learning Transferable Visual Models From Natural Language Supervision
• mixup: Beyond Empirical Risk Minimization
• Improved Regularization of Convolutional Neural Networks with Cutout
• AutoAugment: Learning Augmentation Policies from Data
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Fast is better than free: Revisiting adversarial training
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Generalisation in humans and deep neural networks
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Generalizing to Unseen Domains via Adversarial Data Augmentation
• Robustness May Be at Odds with Accuracy
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• Natural Adversarial Examples
• Motivating the Rules of the Game for Adversarial Example Research
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• RobustBench: a standardized adversarial robustness benchmark
• A Fourier Perspective on Model Robustness in Computer Vision
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing
• First-order Adversarial Vulnerability of Neural Networks and Input Dimension
• Understanding and Improving Fast Adversarial Training
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions
• Transfer of Adversarial Robustness Between Perturbation Types
• With Friends Like These, Who Needs Adversaries?
• Bridging the Performance Gap between FGSM and PGD Adversarial Training

Cited by in corpus (8)

• RobustBench: a standardized adversarial robustness benchmark
• If a Human Can See It, So Should Your System: Reliability Requirements for Machine Vision Components
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training
• Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselines
• A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness
• Defending Against Image Corruptions Through Adversarial Augmentations
• Combining Diverse Feature Priors