Adversarially Robust Generalization Requires More Data

Abstract

Machine learning models are often susceptible to adversarial perturbations of their inputs. Even small perturbations can cause state-of-the-art classifiers with high "standard" accuracy to produce an incorrect prediction with high confidence. To better understand this phenomenon, we study adversarially robust learning from the viewpoint of generalization. We show that already in a simple natural data model, the sample complexity of robust learning can be significantly larger than that of "standard" learning. This gap is information theoretic and holds irrespective of the training algorithm or the model family. We complement our theoretical results with experiments on popular image classification datasets and show that a similar gap exists here as well. We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.

arXiv commentSmall changes for biblatex compatibility

References in corpus (9)

• Explaining and Harnessing Adversarial Examples
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Certified Defenses against Adversarial Examples
• The Space of Transferable Adversarial Examples
• Adversarial Perturbations Against Deep Neural Networks for Malware Classification
• Adversarial Examples for Evaluating Reading Comprehension Systems
• Defensive Distillation is Not Robust to Adversarial Examples
• Analyzing the Robustness of Nearest Neighbors to Adversarial Examples

Cited by in corpus (176)

• On the Opportunities and Risks of Foundation Models
• Adversarial Examples Are Not Bugs, They Are Features
• Robustness May Be at Odds with Accuracy
• Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Adversarial Weight Perturbation Helps Robust Generalization
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Are Labels Required for Improving Adversarial Robustness?
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Adversarial Training Can Hurt Generalization
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Adversarial examples from computational constraints
• Adversarial Training and Robustness for Multiple Perturbations
• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• Rademacher Complexity for Adversarially Robust Generalization
• Fixing Data Augmentation to Improve Adversarial Robustness
• Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• The Risks of Invariant Risk Minimization
• A Survey On Universal Adversarial Attack
• The Pitfalls of Simplicity Bias in Neural Networks
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• A Closer Look at Accuracy vs. Robustness
• A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
• Unlabeled Data Improves Adversarial Robustness
• The Limitations of Adversarial Training and the Blind-Spot Attack
• Security and Privacy Issues in Deep Learning
• ME-Net: Towards Effective Adversarial Robustness with Matrix Estimation
• Excessive Invariance Causes Adversarial Vulnerability
• Convergence of Adversarial Training in Overparametrized Neural Networks
• Robust Local Features for Improving the Generalization of Adversarial Training
• Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
• Overfitting in adversarially robust deep learning
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• VC Classes are Adversarially Robustly Learnable, but Only Improperly
• Adaptative Perturbation Patterns: Realistic Adversarial Learning for Robust Intrusion Detection
• Arms Race in Adversarial Malware Detection: A Survey
• On the Geometry of Adversarial Examples
• PatchGuard: A Provably Robust Defense against Adversarial Patches via Small Receptive Fields and Masking
• On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
• MaxUp: A Simple Way to Improve Generalization of Neural Network Training
• Natural Language Adversarial Defense through Synonym Encoding
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
• The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization
• Cross-Entropy Loss and Low-Rank Features Have Responsibility for Adversarial Examples
• Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
• Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning
• Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
• Universal adversarial examples in speech command classification
• Adversarial Risk Bounds via Function Transformation
• CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting
• Adversarial Neural Pruning with Latent Vulnerability Suppression
• Probabilistic Margins for Instance Reweighting in Adversarial Training
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning
• More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models
• A Marketplace for Data: An Algorithmic Solution
• Towards an Adversarially Robust Normalization Approach
• PAC-learning in the presence of evasion adversaries
• Towards Understanding Adversarial Examples Systematically: Exploring Data Size, Task and Model Factors
• Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models
• A Provable Defense for Deep Residual Networks
• Improving Robustness of Deep-Learning-Based Image Reconstruction
• CheXternal: Generalization of Deep Learning Models for Chest X-ray Interpretation to Photos of Chest X-rays and External Clinical Settings
• A Spectral View of Adversarially Robust Features
• Exploring Memorization in Adversarial Training
• Adversarial symmetric GANs: bridging adversarial samples and adversarial networks
• Training Robust Deep Neural Networks via Adversarial Noise Propagation
• How benign is benign overfitting?
• Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them
• Efficient Adversarial Training with Transferable Adversarial Examples
• On the Generalization Properties of Adversarial Training
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Convergence and Margin of Adversarial Training on Separable Data
• Adversarial Robustness under Long-Tailed Distribution
• Adversarial Machine Learning Phases of Matter
• Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization
• A law of robustness for two-layers neural networks
• When and How Mixup Improves Calibration
• Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distribution
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Computational Limitations in Robust Classification and Win-Win Results
• Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
• Probing the Purview of Neural Networks via Gradient Analysis
• Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training
• Improved Adversarial Robustness via Logit Regularization Methods
• Detecting Adversarial Examples via Neural Fingerprinting
• A Closer Look at Data Bias in Neural Extractive Summarization Models
• Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain
• Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
• Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?
• Robustness via Deep Low-Rank Representations
• Shift Invariance Can Reduce Adversarial Robustness
• Interpreting Neural Networks Using Flip Points
• Robust Attacks against Multiple Classifiers
• Train simultaneously, generalize better: Stability of gradient-based minimax learners
• Adversarial Robustness of Supervised Sparse Coding
• Improving Global Adversarial Robustness Generalization With Adversarially Trained GAN
• SOAR: Second-Order Adversarial Regularization
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• Sharp Statistical Guarantees for Adversarially Robust Gaussian Classification
• A Bayes-Optimal View on Adversarial Examples
• Generating Out of Distribution Adversarial Attack using Latent Space Poisoning
• Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks
• Self-training Avoids Using Spurious Features Under Domain Shift
• Adversarially Robust Low Dimensional Representations
• Recent Advances in Understanding Adversarial Robustness of Deep Neural Networks
• On Visual Hallmarks of Robustness to Adversarial Malware
• Random Directional Attack for Fooling Deep Neural Networks
• Towards Understanding Limitations of Pixel Discretization Against Adversarial Attacks
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Towards Evaluating the Robustness of Neural Networks Learned by Transduction
• AccelAT: A Framework for Accelerating the Adversarial Training of Deep Neural Networks through Accuracy Gradient
• Data Quality Matters For Adversarial Training: An Empirical Study
• Improving Adversarial Robustness via Unlabeled Out-of-Domain Data
• Improving Robustness using Generated Data
• Improving Model Robustness with Latent Distribution Locally and Globally
• ATZSL: Defensive Zero-Shot Recognition in the Presence of Adversaries
• Most ReLU Networks Suffer from ${\ell }^2$ Adversarial Perturbations
• Adversarial Robustness of Deep Learning: Theory, Algorithms, and Applications
• Understanding the Generalization Benefit of Model Invariance from a Data Perspective
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• Asymptotic Behavior of Adversarial Training in Binary Classification
• Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations
• RAIN: A Simple Approach for Robust and Accurate Image Classification Networks
• Statistically Robust Neural Network Classification
• A Survey on Large-scale Machine Learning
• Adversarial Robustness Guarantees for Random Deep Neural Networks
• MEAT: Median-Ensemble Adversarial Training for Improving Robustness and Generalization
• Towards Understanding the Impact of Model Size on Differential Private Classification
• Universal Approximation with Certified Networks
• An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
• Robust Sensible Adversarial Learning of Deep Neural Networks for Image Classification
• Regional Image Perturbation Reduces $L_p$ Norms of Adversarial Examples While Maintaining Model-to-model Transferability
• Robust Classification using Robust Feature Augmentation
• Towards Deep Learning Models Resistant to Large Perturbations
• Learnergy: Energy-based Machine Learners
• A Useful Taxonomy for Adversarial Robustness of Neural Networks
• Augmenting Model Robustness with Transformation-Invariant Attacks
• When are Non-Parametric Methods Robust?
• Query complexity of adversarial attacks
• Enhance Diffusion to Improve Robust Generalization
• Adversarial Attacks on Binary Image Recognition Systems
• Towards the Memorization Effect of Neural Networks in Adversarial Training
• Learning Generalized Spatial-Temporal Deep Feature Representation for No-Reference Video Quality Assessment
• Sample Complexity of Adversarially Robust Linear Classification on Separated Data
• A Law of Robustness for Weight-bounded Neural Networks
• Removing Undesirable Feature Contributions Using Out-of-Distribution Data
• Lower Bounds on Cross-Entropy Loss in the Presence of Test-time Adversaries
• How do SGD hyperparameters in natural training affect adversarial robustness?
• Domain Invariant Adversarial Learning
• State-Reification Networks: Improving Generalization by Modeling the Distribution of Hidden Representations
• A Separation Result Between Data-oblivious and Data-aware Poisoning Attacks
• Robust Deep Neural Networks Inspired by Fuzzy Logic
• Luring of transferable adversarial perturbations in the black-box paradigm
• Classifier-independent Lower-Bounds for Adversarial Robustness
• Large-Scale Multi-Agent Deep FBSDEs
• Covariate Shift in High-Dimensional Random Feature Regression
• Noisy Feature Mixup
• Adversarial Detection by Approximation of Ensemble Boundary
• Improved Algorithms for White-Box Adversarial Streams
• Closing the Gap: Achieving Better Accuracy-Robustness Tradeoffs against Query-Based Attacks
• Adversarial Examples as an Input-Fault Tolerance Problem
• Balancing Robustness and Sensitivity using Feature Contrastive Learning
• Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective
• A Neuro-Inspired Autoencoding Defense Against Adversarial Perturbations
• Estimating Principal Components under Adversarial Perturbations
• Unique properties of adversarially trained linear classifiers on Gaussian data
• Adversarial Examples and Metrics
• Derivation of Information-Theoretically Optimal Adversarial Attacks with Applications to Robust Machine Learning
• Adversarial Training with Stochastic Weight Average
• Adaptive versus Standard Descent Methods and Robustness Against Adversarial Examples