Improving Robustness using Generated Data

Abstract

Recent work argues that robust training requires substantially larger datasets than those required for standard classification. On CIFAR-10 and CIFAR-100, this translates into a sizable robust-accuracy gap between models trained solely on data from the original training set and those trained with additional data extracted from the "80 Million Tiny Images" dataset (TI-80M). In this paper, we explore how generative models trained solely on the original training set can be leveraged to artificially increase the size of the original training set and improve adversarial robustness to ${\ell }_p$ norm-bounded perturbations. We identify the sufficient conditions under which incorporating additional generated data can improve robustness, and demonstrate that it is possible to significantly reduce the robust-accuracy gap to models trained with additional real data. Surprisingly, we even show that even the addition of non-realistic random data (generated by Gaussian sampling) can improve robustness. We evaluate our approach on CIFAR-10, CIFAR-100, SVHN and TinyImageNet against ${\ell }_{\infty }$ and ${\ell }_2$ norm-bounded perturbations of size $\epsilon =8/255$ and $\epsilon =128/255$, respectively. We show large absolute improvements in robust accuracy compared to previous state-of-the-art methods. Against ${\ell }_{\infty }$ norm-bounded perturbations of size $\epsilon =8/255$, our models achieve 66.10% and 33.49% robust accuracy on CIFAR-10 and CIFAR-100, respectively (improving upon the state-of-the-art by +8.96% and +3.29%). Against ${\ell }_2$ norm-bounded perturbations of size $\epsilon =128/255$, our model achieves 78.31% on CIFAR-10 (+3.81%). These results beat most prior works that use external data.

Accepted at NeurIPS 2021; Added ImageNet results

References in corpus (17)

• Improved Regularization of Convolutional Neural Networks with Cutout
• Diffusion Models Beat GANs on Image Synthesis
• Theoretically Principled Trade-off between Robustness and Accuracy
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• GANSpace: Discovering Interpretable GAN Controls
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• On Adaptive Attacks to Adversarial Example Defenses
• RobustBench: a standardized adversarial robustness benchmark
• Controlling generative models with continuous factors of variations
• Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
• An Alternative Surrogate Loss for PGD-based Adversarial Testing
• On Adversarial Bias and the Robustness of Fair Machine Learning
• Scale MLPerf-0.6 models on Google TPU-v3 Pods
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Lower Bounds on Adversarial Robustness from Optimal Transport
• Learnable Boundary Guided Adversarial Training
• Bag of Tricks for Adversarial Training