Testing Robustness Against Unforeseen Adversaries

Abstract

Adversarial robustness research primarily focuses on L_p perturbations, and most defenses are developed with identical training-time and test-time adversaries. However, in real-world applications developers are unlikely to have access to the full range of attacks or corruptions their system will face. Furthermore, worst-case inputs are likely to be diverse and need not be constrained to the L_p ball. To narrow in on this discrepancy between research and reality we introduce ImageNet-UA, a framework for evaluating model robustness against a range of unforeseen adversaries, including eighteen new non-L_p attacks. To perform well on ImageNet-UA, defenses must overcome a generalization gap and be robust to a diverse attacks not encountered during training. In extensive experiments, we find that existing robustness measures do not capture unforeseen robustness, that standard robustness techniques are beat by alternative training strategies, and that novel methods can improve unforeseen robustness. We present ImageNet-UA as a useful tool for the community for improving the worst-case behavior of machine learning systems.

arXiv commentDatasets available at https://github.com/centerforaisafety/adversarial-corruptions

References in corpus (32)

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Synthesizing Robust Adversarial Examples
• On Evaluating Adversarial Robustness
• Robust Physical-World Attacks on Deep Learning Models
• Aggregated Residual Transformations for Deep Neural Networks
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• Spatially Transformed Adversarial Examples
• Know What You Don't Know: Unanswerable Questions for SQuAD
• A General Framework for Adversarial Examples with Objectives
• Adversarial Robustness through Local Linearization
• Natural Adversarial Examples
• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
• Motivating the Rules of the Game for Adversarial Example Research
• Examining the Impact of Blur on Recognition by Convolutional Networks
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Constructing Unrestricted Adversarial Examples with Generative Models
• DeepFool: a simple and accurate method to fool deep neural networks
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Attacking the Madry Defense Model with $L_1$-based Adversarial Examples
• Feature Denoising for Improving Adversarial Robustness
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing
• Quantifying Perceptual Distortion of Adversarial Examples
• L2-Nonexpansive Neural Networks
• Robustness properties of Facebook's ResNeXt WSL models
• Gradient Masking Causes CLEVER to Overestimate Adversarial Perturbation Size
• Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
• A Research Agenda: Dynamic Models to Defend Against Correlated Attacks
• Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distance
• Sensitivity of Deep Convolutional Networks to Gabor Noise

Cited by in corpus (36)

• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Measuring Robustness to Natural Distribution Shifts in Image Classification
• RobustBench: a standardized adversarial robustness benchmark
• Unsolved Problems in ML Safety
• Robust Pre-Training by Adversarial Contrastive Learning
• Unrestricted Adversarial Examples via Semantic Manipulation
• Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
• Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
• AI Safety for Everyone
• Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study
• Unadversarial Examples: Designing Objects for Robust Vision
• Mutual Adversarial Training: Learning together is better than going alone
• Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp Adversarial Attacks
• SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing
• Consistency Regularization for Certified Robustness of Smoothed Classifiers
• Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
• BREEDS: Benchmarks for Subpopulation Shift
• 3DB: A Framework for Debugging Computer Vision Models
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Evaluating Robustness to Context-Sensitive Feature Perturbations of Different Granularities
• Identifying Statistical Bias in Dataset Replication
• Efficiently Learning Adversarially Robust Halfspaces with Noise
• Beyond Perturbations: Learning Guarantees with Arbitrary Adversarial Test Examples
• Towards Defending Multiple ${\ell }_p$-norm Bounded Adversarial Perturbations via Gated Batch Normalization
• Towards an Awareness of Time Series Anomaly Detection Models' Adversarial Vulnerability
• Shift Equivariance in Object Detection
• Towards Corruption-Agnostic Robust Domain Adaptation
• A Survey on Trust Metrics for Autonomous Robotic Systems
• Why Accuracy Is Not Enough: The Need for Consistency in Object Detection
• ATRO: Adversarial Training with a Rejection Option
• Adversarially Robust Learning with Unknown Perturbation Sets
• Adversarial and Natural Perturbations for General Robustness
• Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Training
• Interpretability Aware Model Training to Improve Robustness against Out-of-Distribution Magnetic Resonance Images in Alzheimer's Disease Classification
• Understanding Robustness in Teacher-Student Setting: A New Perspective