Adversarial Examples Are a Natural Consequence of Test Error in Noise

Abstract

Over the last few years, the phenomenon of adversarial examples --- maliciously constructed inputs that fool trained machine learning models --- has captured the attention of the research community, especially when the adversary is restricted to small modifications of a correctly handled input. Less surprisingly, image classifiers also lack human-level performance on randomly corrupted images, such as images with additive Gaussian noise. In this paper we provide both empirical and theoretical evidence that these are two manifestations of the same underlying phenomenon, establishing close connections between the adversarial robustness and corruption robustness research programs. This suggests that improving adversarial robustness should go hand in hand with improving performance in the presence of more general and realistic image corruptions. Based on our results we recommend that future adversarial defenses consider evaluating the robustness of their methods to distributional shift with benchmarks such as Imagenet-C.

References in corpus (17)

• Wide Residual Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• AutoAugment: Learning Augmentation Policies from Data
• Adversarially Robust Generalization Requires More Data
• Countering Adversarial Images using Input Transformations
• A study of the effect of JPG compression on adversarial images
• Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression
• Mitigating Adversarial Effects Through Randomization
• Why do deep convolutional networks generalize so poorly to small image transformations?
• Motivating the Rules of the Game for Adversarial Example Research
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Attacking the Madry Defense Model with $L_1$-based Adversarial Examples
• Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression
• A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions
• Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples

Cited by in corpus (69)

• On Evaluating Adversarial Robustness
• Adversarial Examples Are Not Bugs, They Are Features
• Improving robustness against common corruptions by covariate shift adaptation
• The Origins and Prevalence of Texture Bias in Convolutional Neural Networks
• RobustBench: a standardized adversarial robustness benchmark
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Label-Only Membership Inference Attacks
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
• A Fourier Perspective on Model Robustness in Computer Vision
• Testing Robustness Against Unforeseen Adversaries
• Boosting Adversarial Training with Hypersphere Embedding
• Conditional GAN for timeseries generation
• Batch Normalization is a Cause of Adversarial Vulnerability
• Affinity and Diversity: Quantifying Mechanisms of Data Augmentation
• Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input Transformation
• Adversarial Distributional Training for Robust Deep Learning
• Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracy
• Consistency Regularization for Certified Robustness of Smoothed Classifiers
• BREEDS: Benchmarks for Subpopulation Shift
• Towards an Adversarially Robust Normalization Approach
• On the effectiveness of adversarial training against common corruptions
• Towards Understanding Adversarial Examples Systematically: Exploring Data Size, Task and Model Factors
• Investigating Vulnerability to Adversarial Examples on Multimodal Data Fusion in Deep Learning
• 3DB: A Framework for Debugging Computer Vision Models
• Using learned optimizers to make models robust to input noise
• Convergence and Margin of Adversarial Training on Separable Data
• Lower Bounds for Adversarially Robust PAC Learning
• PDA: Progressive Data Augmentation for General Robustness of Deep Neural Networks
• Jacobian Adversarially Regularized Networks for Robustness
• Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attacks
• Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
• Stateful Detection of Black-Box Adversarial Attacks
• Robustness in Compressed Neural Networks for Object Detection
• Understanding Adversarial Examples from the Mutual Influence of Images and Perturbations
• Training on Test Data with Bayesian Adaptation for Covariate Shift
• Predicting with High Correlation Features
• ATRO: Adversarial Training with a Rejection Option
• Statistically Robust Neural Network Classification
• Defective Convolutional Networks
• Robustness via Cross-Domain Ensembles
• Adversarial Heart Attack: Neural Networks Fooled to Segment Heart Symbols in Chest X-Ray Images
• Natural Perturbed Training for General Robustness of Neural Network Classifiers
• Towards Deep Learning Models Resistant to Large Perturbations
• Utilizing Network Properties to Detect Erroneous Inputs
• Adversarial Robustness Guarantees for Random Deep Neural Networks
• Adversarial and Natural Perturbations for General Robustness
• MixDefense: A Defense-in-Depth Framework for Adversarial Example Detection Based on Statistical and Semantic Analysis
• Golden Grain: Building a Secure and Decentralized Model Marketplace for MLaaS
• Heat and Blur: An Effective and Fast Defense Against Adversarial Examples
• Robusta: Robust AutoML for Feature Selection via Reinforcement Learning
• Adversarial robustness via stochastic regularization of neural activation sensitivity
• A general framework for defining and optimizing robustness
• Countering Adversarial Examples: Combining Input Transformation and Noisy Training
• Multi-way Encoding for Robustness
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality
• Adversarial Data Encryption
• Robustifying ${\ell }_{\infty }$ Adversarial Training to the Union of Perturbation Models
• Improved Robustness of Vision Transformer via PreLayerNorm in Patch Embedding
• Verifying the Causes of Adversarial Examples
• Oriole: Thwarting Privacy against Trustworthy Deep Learning Models
• Gödel's Sentence Is An Adversarial Example But Unsolvable
• Robust Deep Neural Networks Inspired by Fuzzy Logic
• ODE guided Neural Data Augmentation Techniques for Time Series Data and its Benefits on Robustness
• Combining Diverse Feature Priors
• Constant Random Perturbations Provide Adversarial Robustness with Minimal Effect on Accuracy
• Reject Illegal Inputs with Generative Classifier Derived from Any Discriminative Classifier
• Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective
• Classifier-independent Lower-Bounds for Adversarial Robustness
• Contextual Fusion For Adversarial Robustness