Robustness May Be at Odds with Accuracy

Abstract

We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization. Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of standard accuracy. We demonstrate that this trade-off between the standard accuracy of a model and its robustness to adversarial perturbations provably exists in a fairly simple and natural setting. These findings also corroborate a similar phenomenon observed empirically in more complex settings. Further, we argue that this phenomenon is a consequence of robust classifiers learning fundamentally different feature representations than standard classifiers. These differences, in particular, seem to result in unexpected benefits: the representations learned by robust models tend to align better with salient data characteristics and human perception.

arXiv commentICLR'19

References in corpus (22)

• Auto-Encoding Variational Bayes
• Understanding Neural Networks Through Deep Visualization
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Robust Physical-World Attacks on Deep Learning Models
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• Shake-Shake regularization
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Exploring the Landscape of Spatial Robustness
• Spatially Transformed Adversarial Examples
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• A Dual Approach to Scalable Verification of Deep Networks
• Adversarial vulnerability for any classifier
• Adversarial examples from computational constraints
• Robustness of classifiers: from adversarial to random noise
• Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
• Analyzing the Robustness of Nearest Neighbors to Adversarial Examples
• Training verified learners with learned verifiers
• Adversarial Spheres

Cited by in corpus (174)

• Motivating the Rules of the Game for Adversarial Example Research
• Training robust neural networks using Lipschitz bounds
• Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors
• Rademacher Complexity for Adversarially Robust Generalization
• RobustBench: a standardized adversarial robustness benchmark
• Adversarial Attacks Against Medical Deep Learning Systems
• Interpreting and Improving Adversarial Robustness of Deep Neural Networks with Neuron Sensitivity
• Feature Denoising for Improving Adversarial Robustness
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Semantic Robustness of Models of Source Code
• Adversarial Robustness May Be at Odds With Simplicity
• Adversarial Attack and Defense on Point Sets
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Sorting out Lipschitz function approximation
• DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensembles
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack
• Contrastive Learning with Adversarial Examples
• Learning from others' mistakes: Avoiding dataset biases without modeling them
• Adversarial Examples in Modern Machine Learning: A Review
• Adversarial Explanations for Understanding Image Classification Decisions and Improved Neural Network Robustness
• Adversarial Examples Improve Image Recognition
• Fawkes: Protecting Privacy against Unauthorized Deep Learning Models
• Understanding and Improving Fast Adversarial Training
• Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
• Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalization
• Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems
• On Robustness of Neural Ordinary Differential Equations
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• Bad Global Minima Exist and SGD Can Reach Them
• Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
• Transfer of Machine Learning Fairness across Domains
• Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• To be Robust or to be Fair: Towards Fairness in Adversarial Training
• How Does Mixup Help With Robustness and Generalization?
• COPYCAT: Practical Adversarial Attacks on Visualization-Based Malware Detection
• There is Limited Correlation between Coverage and Robustness for Deep Neural Networks
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning
• Deep Nets: What have they ever done for Vision?
• Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
• White Paper Machine Learning in Certified Systems
• Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving
• Towards Understanding Adversarial Examples Systematically: Exploring Data Size, Task and Model Factors
• On the effectiveness of adversarial training against common corruptions
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• Certifiably Robust Interpretation in Deep Learning
• Intriguing properties of adversarial training at scale
• Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
• ML-LOO: Detecting Adversarial Examples with Feature Attribution
• The Search for Sparse, Robust Neural Networks
• Training Robust Deep Neural Networks via Adversarial Noise Propagation
• Adversarial symmetric GANs: bridging adversarial samples and adversarial networks
• Adversarial Attacks on Monocular Depth Estimation
• An Empirical Analysis of the Impact of Data Augmentation on Knowledge Distillation
• Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical World
• Featurized Bidirectional GAN: Adversarial Defense via Adversarially Learned Semantic Inference
• Bridging the Performance Gap between FGSM and PGD Adversarial Training
• Adversarial Robustness via Label-Smoothing
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Convergence and Margin of Adversarial Training on Separable Data
• Neural Architecture Dilation for Adversarial Robustness
• Robust or Private? Adversarial Training Makes Models More Vulnerable to Privacy Attacks
• On Certifying Non-uniform Bound against Adversarial Attacks
• Bit Error Robustness for Energy-Efficient DNN Accelerators
• Jacobian Adversarially Regularized Networks for Robustness
• Adversarial Visual Robustness by Causal Intervention
• Adversarial Robustness Guarantees for Classification with Gaussian Processes
• Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distribution
• On Data Augmentation and Adversarial Risk: An Empirical Analysis
• Rethinking Natural Adversarial Examples for Classification Models
• Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
• RobFR: Benchmarking Adversarial Robustness on Face Recognition
• OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data
• DAmageNet: A Universal Adversarial Dataset
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Sharp Statistical Guarantees for Adversarially Robust Gaussian Classification
• Adversarial Robustness of Supervised Sparse Coding
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• DeepBlur: A Simple and Effective Method for Natural Image Obfuscation
• Interpreting Adversarial Examples with Attributes
• Feature Prioritization and Regularization Improve Standard Accuracy and Adversarial Robustness
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency
• Improving Certified Robustness via Statistical Learning with Logical Reasoning
• Enhancing Adversarial Defense by k-Winners-Take-All
• Calibration and Consistency of Adversarial Surrogate Losses
• Self-training Avoids Using Spurious Features Under Domain Shift
• A Method for Computing Class-wise Universal Adversarial Perturbations
• Adversarially Robust Low Dimensional Representations
• On the Benefits of Models with Perceptually-Aligned Gradients
• Design and Interpretation of Universal Adversarial Patches in Face Detection
• Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks
• Data Quality Matters For Adversarial Training: An Empirical Study
• No Surprises: Training Robust Lung Nodule Detection for Low-Dose CT Scans by Augmenting with Adversarial Attacks
• CAAD 2018: Generating Transferable Adversarial Examples
• Improving Robustness using Generated Data
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Random Shadows and Highlights: A new data augmentation method for extreme lighting conditions
• Asymptotic Behavior of Adversarial Training in Binary Classification
• Improved Adversarial Training via Learned Optimizer
• Designing Adversarially Resilient Classifiers using Resilient Feature Engineering
• Analytical Moment Regularizer for Gaussian Robust Networks
• A Survey on Trust Metrics for Autonomous Robotic Systems
• Architecture Selection via the Trade-off Between Accuracy and Robustness
• A Singular Value Perspective on Model Robustness
• Purifying Adversarial Perturbation with Adversarially Trained Auto-encoders
• Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search
• Predicting with High Correlation Features
• RAIN: A Simple Approach for Robust and Accurate Image Classification Networks
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• On the human-recognizability phenomenon of adversarially trained deep image classifiers
• Adversarial Robustness for Unsupervised Domain Adaptation
• Exploring Transferable and Robust Adversarial Perturbation Generation from the Perspective of Network Hierarchy
• Fundamental Tradeoffs in Distributionally Adversarial Training
• Natural Perturbed Training for General Robustness of Neural Network Classifiers
• ZeroGrad : Mitigating and Explaining Catastrophic Overfitting in FGSM Adversarial Training
• Towards Deep Learning Models Resistant to Large Perturbations
• Blind Adversarial Training: Balance Accuracy and Robustness
• What it Thinks is Important is Important: Robustness Transfers through Input Gradients
• Defective Convolutional Networks
• Adversarial and Natural Perturbations for General Robustness
• Enabling Data Diversity: Efficient Automatic Augmentation via Regularized Adversarial Training
• Improving the trustworthiness of image classification models by utilizing bounding-box annotations
• An Experimental Study of Semantic Continuity for Deep Learning Models
• Removing Undesirable Feature Contributions Using Out-of-Distribution Data
• Adversarial Training for Face Recognition Systems using Contrastive Adversarial Learning and Triplet Loss Fine-tuning
• Robusta: Robust AutoML for Feature Selection via Reinforcement Learning
• Domain Invariant Adversarial Learning
• Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs
• Deep Latent Defence
• Out-of-Distribution Generalization in Kernel Regression
• On the transferability of adversarial examples between convex and 01 loss models
• A principled approach for generating adversarial images under non-smooth dissimilarity metrics
• Robust Generalization of Quadratic Neural Networks via Function Identification
• Adversarial Robustness for Machine Learning Cyber Defenses Using Log Data
• A Useful Taxonomy for Adversarial Robustness of Neural Networks
• DISCO Verification: Division of Input Space into COnvex polytopes for neural network verification
• Towards the Memorization Effect of Neural Networks in Adversarial Training
• Augmenting Model Robustness with Transformation-Invariant Attacks
• Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Training
• Posterior Differential Regularization with f-divergence for Improving Model Robustness
• Bio-inspired Robustness: A Review
• Robust Regularization with Adversarial Labelling of Perturbed Samples
• ODE guided Neural Data Augmentation Techniques for Time Series Data and its Benefits on Robustness
• Decoder-free Robustness Disentanglement without (Additional) Supervision
• On Connections between Regularizations for Improving DNN Robustness
• Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness
• What's the relationship between CNNs and communication systems?
• An Integrated Approach to Produce Robust Models with High Efficiency
• Built-in Elastic Transformations for Improved Robustness
• On Generalization of Graph Autoencoders with Adversarial Training
• Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses
• Understanding Adversarial Behavior of DNNs by Disentangling Non-Robust and Robust Components in Performance Metric
• Notes on Margin Training and Margin p-Values for Deep Neural Network Classifiers
• Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples
• Investigating the Robustness of Artificial Intelligent Algorithms with Mixture Experiments
• Understanding Misclassifications by Attributes
• Understanding Robustness in Teacher-Student Setting: A New Perspective
• On Robustness of Neural Semantic Parsers
• A Separation Result Between Data-oblivious and Data-aware Poisoning Attacks
• A general framework for defining and optimizing robustness
• Empirical Upper Bound, Error Diagnosis and Invariance Analysis of Modern Object Detectors
• Achieving Adversarial Robustness Requires An Active Teacher
• Improving the Certified Robustness of Neural Networks via Consistency Regularization
• Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More
• Noisy Feature Mixup
• One Man's Trash is Another Man's Treasure: Resisting Adversarial Examples by Adversarial Examples
• Adversarial Training for EM Classification Networks
• Training Efficiency and Robustness in Deep Learning
• NeuralDP Differentially private neural networks by design
• Estimating Principal Components under Adversarial Perturbations
• Robust Deep Neural Networks Inspired by Fuzzy Logic
• Unique properties of adversarially trained linear classifiers on Gaussian data
• Balancing Robustness and Sensitivity using Feature Contrastive Learning