Adversarial Examples Are Not Bugs, They Are Features

Abstract

Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear. We demonstrate that adversarial examples can be directly attributed to the presence of non-robust features: features derived from patterns in the data distribution that are highly predictive, yet brittle and incomprehensible to humans. After capturing these features within a theoretical framework, we establish their widespread existence in standard datasets. Finally, we present a simple setting where we can rigorously tie the phenomena we observe in practice to a misalignment between the (human-specified) notion of robustness and the inherent geometry of the data.

References in corpus (25)

• Distilling the Knowledge in a Neural Network
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• SmoothGrad: removing noise by adding noise
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• On Evaluating Adversarial Robustness
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• The Space of Transferable Adversarial Examples
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Adversarial vulnerability for any classifier
• A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Adversarial examples from computational constraints
• Do CIFAR-10 Classifiers Generalize to CIFAR-10?
• Robustness of classifiers: from adversarial to random noise
• Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Adversarial Robustness May Be at Odds With Simplicity
• A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
• Adversarial Spheres
• Bridging Adversarial Robustness and Gradient Interpretability
• Revisiting Adversarial Risk
• A Geometric Perspective on the Transferability of Adversarial Directions

Cited by in corpus (120)

• Shortcut Learning in Deep Neural Networks
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds
• The Origins and Prevalence of Texture Bias in Convolutional Neural Networks
• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Going in circles is the way forward: the role of recurrence in visual inference
• The Conditional Entropy Bottleneck
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
• Adversarial Policies: Attacking Deep Reinforcement Learning
• A Fourier Perspective on Model Robustness in Computer Vision
• A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directions
• Towards quantum enhanced adversarial robustness in machine learning
• Robust Local Features for Improving the Generalization of Adversarial Training
• Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges
• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• A neural network walks into a lab: towards using deep nets as models for human behavior
• Label-Consistent Backdoor Attacks
• Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization
• Recent Advances in Adversarial Training for Adversarial Robustness
• High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
• Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalization
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• Large batch size training of neural networks with adversarial training and second-order information
• Availability Attacks Create Shortcuts
• Zero-shot Knowledge Transfer via Adversarial Belief Matching
• Adversarial Examples Make Strong Poisons
• If MaxEnt RL is the Answer, What is the Question?
• AI Safety for Everyone
• The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization
• Maximum Entropy RL (Provably) Solves Some Robust RL Problems
• BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations
• Towards a Robust Deep Neural Network in Texts: A Survey
• Exposing Previously Undetectable Faults in Deep Neural Networks
• Drastic Circuit Depth Reductions with Preserved Adversarial Robustness by Approximate Encoding for Quantum Machine Learning
• Universal adversarial examples in speech command classification
• Data-driven Identification of 2D Partial Differential Equations using extracted physical features
• When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures
• Beneficial Perturbation Network for designing general adaptive artificial intelligence systems
• Adversarial Machine Learning in Wireless Communications using RF Data: A Review
• Towards Assessing the Synthetic-to-Measured Adversarial Vulnerability of SAR ATR
• Intriguing properties of adversarial training at scale
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Artificial Intelligence Strategies for National Security and Safety Standards
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• CEB Improves Model Robustness
• Efficient Adversarial Training with Transferable Adversarial Examples
• Input Validation for Neural Networks via Runtime Local Robustness Verification
• In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness
• Now You See It, Now You Dont: Adversarial Vulnerabilities in Computational Pathology
• To Transfer or Not to Transfer: Misclassification Attacks Against Transfer Learned Text Classifiers
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Investigating Decision Boundaries of Trained Neural Networks
• Instance Correction for Learning with Open-set Noisy Labels
• Adversarial Attack and Defense in Deep Ranking
• The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples
• An Empirical Evaluation of Adversarial Robustness under Transfer Learning
• Semantics Preserving Adversarial Learning
• Recent Advances in Understanding Adversarial Robustness of Deep Neural Networks
• Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
• No One Representation to Rule Them All: Overlapping Features of Training Methods
• Random Directional Attack for Fooling Deep Neural Networks
• Adversarial shape perturbations on 3D point clouds
• Rearchitecting Classification Frameworks For Increased Robustness
• Internal Wasserstein Distance for Adversarial Attack and Defense
• Calibrated neighborhood aware confidence measure for deep metric learning
• An empirical study of pretrained representations for few-shot classification
• Cost-effective Interactive Attention Learning with Neural Attention Processes
• The Dimpled Manifold Model of Adversarial Examples in Machine Learning
• Training Meta-Surrogate Model for Transferable Adversarial Attack
• Self-supervised Adversarial Training
• A Self-supervised Approach for Adversarial Robustness
• Auditing and Debugging Deep Learning Models via Decision Boundaries: Individual-level and Group-level Analysis
• Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately
• Data augmentation and image understanding
• How Does Adversarial Fine-Tuning Benefit BERT?
• Understanding and Diagnosing Vulnerability under Adversarial Attacks
• Recurrent Attention Model with Log-Polar Mapping is Robust against Adversarial Attacks
• Reliable Graph Neural Network Explanations Through Adversarial Training
• The Principle of Unchanged Optimality in Reinforcement Learning Generalization
• Robust Classification using Robust Feature Augmentation
• Beneficial Perturbations Network for Defending Adversarial Examples
• Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness
• Learning to Find Correlated Features by Maximizing Information Flow in Convolutional Neural Networks
• Learning Diverse Representations for Fast Adaptation to Distribution Shift
• Defective Convolutional Networks
• Smoothed Inference for Adversarially-Trained Models
• A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples
• Out-of-Distribution Generalization in Kernel Regression
• Do Input Gradients Highlight Discriminative Features?
• Generating Adversarial Inputs Using A Black-box Differential Technique
• Adversarial training applied to Convolutional Neural Network for photometric redshift predictions
• A Useful Taxonomy for Adversarial Robustness of Neural Networks
• Heat and Blur: An Effective and Fast Defense Against Adversarial Examples
• Robust Generalization of Quadratic Neural Networks via Function Identification
• Provably robust deep generative models
• Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Training
• Orthogonal Deep Models As Defense Against Black-Box Attacks
• Less is More: Feature Selection for Adversarial Robustness with Compressive Counter-Adversarial Attacks
• Attack to Fool and Explain Deep Networks
• Delving into the pixels of adversarial samples
• Identifying Layers Susceptible to Adversarial Attacks
• Towards Explaining Adversarial Examples Phenomenon in Artificial Neural Networks
• Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective
• And/or trade-off in artificial neurons: impact on adversarial robustness
• Context Dependent Semantic Parsing: A Survey
• Adversarial Training for EM Classification Networks
• Verifying the Causes of Adversarial Examples
• Adversarial Examples and the Deeper Riddle of Induction: The Need for a Theory of Artifacts in Deep Learning
• Understanding (Non-)Robust Feature Disentanglement and the Relationship Between Low- and High-Dimensional Adversarial Attacks
• HRFA: High-Resolution Feature-based Attack
• Robustifying ${\ell }_{\infty }$ Adversarial Training to the Union of Perturbation Models
• Decoder-free Robustness Disentanglement without (Additional) Supervision
• Learning to Disentangle Robust and Vulnerable Features for Adversarial Detection
• Analyzing the Dependency of ConvNets on Spatial Information
• Gödel's Sentence Is An Adversarial Example But Unsolvable
• Revisiting Ensembles in an Adversarial Context: Improving Natural Accuracy
• Understanding Adversarial Behavior of DNNs by Disentangling Non-Robust and Robust Components in Performance Metric
• The Probabilistic Fault Tolerance of Neural Networks in the Continuous Limit