Improving Adversarial Robustness via Unlabeled Out-of-Domain Data

Abstract

Data augmentation by incorporating cheap unlabeled data from multiple domains is a powerful way to improve prediction especially when there is limited labeled data. In this work, we investigate how adversarial robustness can be enhanced by leveraging out-of-domain unlabeled data. We demonstrate that for broad classes of distributions and classifiers, there exists a sample complexity gap between standard and robust classification. We quantify to what degree this gap can be bridged via leveraging unlabeled samples from a shifted domain by providing both upper and lower bounds. Moreover, we show settings where we achieve better adversarial robustness when the unlabeled data come from a shifted domain rather than the same domain as the labeled data. We also investigate how to leverage out-of-domain data when some structural information, such as sparsity, is shared between labeled and unlabeled domains. Experimentally, we augment two object recognition datasets (CIFAR-10 and SVHN) with easy to obtain and unlabeled out-of-domain data and demonstrate substantial improvement in the model's robustness against ${\ell }_{\infty }$ adversarial attacks on the original domain.

arXiv commentAccepted by AISTATS 2021, and selected as the oral presentation (top 3% of submissions)

References in corpus (22)

• Wide Residual Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Temporal Ensembling for Semi-Supervised Learning
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Synthesizing Robust Adversarial Examples
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Countering Adversarial Images using Input Transformations
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Machine Learning at Scale
• Are Labels Required for Improving Adversarial Robustness?
• Adversarial examples from computational constraints
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• How Does Mixup Help With Robustness and Generalization?
• Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality
• Computational Limitations in Robust Classification and Win-Win Results
• Enhancing Certified Robustness via Smoothed Weighted Ensembling
• Towards Understanding the Dynamics of the First-Order Adversaries
• Interpreting Robust Optimization via Adversarial Influence Functions