Natural Adversarial Examples

Abstract

We introduce two challenging datasets that reliably cause machine learning model performance to substantially degrade. The datasets are collected with a simple adversarial filtration technique to create datasets with limited spurious cues. Our datasets' real-world, unmodified examples transfer to various unseen models reliably, demonstrating that computer vision models have shared weaknesses. The first dataset is called ImageNet-A and is like the ImageNet test set, but it is far more challenging for existing models. We also curate an adversarial out-of-distribution detection dataset called ImageNet-O, which is the first out-of-distribution detection dataset created for ImageNet models. On ImageNet-A a DenseNet-121 obtains around 2% accuracy, an accuracy drop of approximately 90%, and its out-of-distribution detection performance on ImageNet-O is near random chance levels. We find that existing data augmentation techniques hardly boost performance, and using other public training datasets provides improvements that are limited. However, we find that improvements to computer vision architectures provide a promising path towards robust models.

arXiv commentCVPR 2021; dataset and code available at https://github.com/hendrycks/natural-adv-examples

References in corpus (36)

• Deep Residual Learning for Image Recognition
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Res2Net: A New Multi-scale Backbone Architecture
• Improved Regularization of Convolutional Neural Networks with Cutout
• Self-Attention Generative Adversarial Networks
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Shortcut Learning in Deep Neural Networks
• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Synthesizing Robust Adversarial Examples
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
• Fast is better than free: Revisiting adversarial training
• Generalisation in humans and deep neural networks
• Aggregated Residual Transformations for Deep Neural Networks
• Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• Learning Robust Global Representations by Penalizing Local Predictive Power
• Adversarial Transformation Networks: Learning to Generate Adversarial Examples
• Scaling Out-of-Distribution Detection for Real-World Settings
• Motivating the Rules of the Game for Adversarial Example Research
• Examining the Impact of Blur on Recognition by Convolutional Networks
• Constructing Unrestricted Adversarial Examples with Generative Models
• Aligning AI With Shared Human Values
• DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
• Attacking the Madry Defense Model with $L_1$-based Adversarial Examples
• A Fourier Perspective on Model Robustness in Computer Vision
• Feature Denoising for Improving Adversarial Robustness
• Unrestricted Adversarial Examples
• HellaSwag: Can a Machine Really Finish Your Sentence?
• Abductive Commonsense Reasoning
• Dual Path Networks
• On Feature Normalization and Data Augmentation
• Towards neural networks that provably know when they don't know
• Identifying Statistical Bias in Dataset Replication

Cited by in corpus (80)

• Learning Transferable Visual Models From Natural Language Supervision
• On the Opportunities and Risks of Foundation Models
• Shortcut Learning in Deep Neural Networks
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• Adversarial Reciprocal Points Learning for Open Set Recognition
• Self-training with Noisy Student improves ImageNet classification
• Improving robustness against common corruptions by covariate shift adaptation
• FMix: Enhancing Mixed Sample Data Augmentation
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
• What Do Compressed Deep Neural Networks Forget?
• MEMO: Test Time Robustness via Adaptation and Augmentation
• The Pitfalls of Simplicity Bias in Neural Networks
• An Investigation of Why Overparameterization Exacerbates Spurious Correlations
• AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights
• Testing Robustness Against Unforeseen Adversaries
• Measuring and modeling the motor system with machine learning
• Rethinking Spatial Dimensions of Vision Transformers
• Long-Short Transformer: Efficient Transformers for Language and Vision
• Adversarial Examples Improve Image Recognition
• Out-of-distribution Detection in Classifiers via Generation
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
• MoPro: Webly Supervised Learning with Momentum Prototypes
• TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devices
• Pretrained Transformers Improve Out-of-Distribution Robustness
• On Feature Normalization and Data Augmentation
• Exposing Previously Undetectable Faults in Deep Neural Networks
• Robustness properties of Facebook's ResNeXt WSL models
• On the effectiveness of adversarial training against common corruptions
• 3DB: A Framework for Debugging Computer Vision Models
• Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation
• A Little Fog for a Large Turn
• Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training
• How benign is benign overfitting?
• On Mixup Regularization
• Evaluating Robustness to Context-Sensitive Feature Perturbations of Different Granularities
• Towards Robust Vision Transformer
• Deep Ensembles for Low-Data Transfer Learning
• A Critical Evaluation of Open-World Machine Learning
• I Am Going MAD: Maximum Discrepancy Competition for Comparing Classifiers Adaptively
• Quizbowl: The Case for Incremental Question Answering
• DAmageNet: A Universal Adversarial Dataset
• Rethinking Natural Adversarial Examples for Classification Models
• Towards Resistant Audio Adversarial Examples
• Unrestricted Adversarial Attacks on ImageNet Competition
• Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels
• The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks
• Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
• Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimation
• Shape-Texture Debiased Neural Network Training
• Calibrated neighborhood aware confidence measure for deep metric learning
• Towards Corruption-Agnostic Robust Domain Adaptation
• Adversarially Optimized Mixup for Robust Classification
• Data Quality Matters For Adversarial Training: An Empirical Study
• Our Evaluation Metric Needs an Update to Encourage Generalization
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Why do classifier accuracies show linear trends under distribution shift?
• Improved Adversarial Training via Learned Optimizer
• A Singular Value Perspective on Model Robustness
• Understanding the effect of sparsity on neural networks robustness
• Defending Against Image Corruptions Through Adversarial Augmentations
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation
• Utilizing Network Properties to Detect Erroneous Inputs
• ImageNet-trained deep neural network exhibits illusion-like response to the Scintillating Grid
• Can we have it all? On the Trade-off between Spatial and Adversarial Robustness of Neural Networks
• Sparse MoEs meet Efficient Ensembles
• Why Do Better Loss Functions Lead to Less Transferable Features?
• Dissecting Image Crops
• Natural Adversarial Objects
• Personalizing Pre-trained Models
• Explainable AI for Natural Adversarial Images
• Out-of-Distribution Robustness in Deep Learning Compression
• Fast Concept Mapping: The Emergence of Human Abilities in Artificial Neural Networks when Learning Embodied and Self-Supervised
• Reproduction of Lateral Inhibition-Inspired Convolutional Neural Network for Visual Attention and Saliency Detection
• Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations
• RC2020 Report: Learning De-biased Representations with Biased Representations
• Multilayer Dense Connections for Hierarchical Concept Classification
• Why Out-of-distribution Detection in CNNs Does Not Like Mahalanobis -- and What to Use Instead
• Gödel's Sentence Is An Adversarial Example But Unsolvable
• Evaluation of Out-of-Distribution Detection Performance of Self-Supervised Learning in a Controllable Environment