On Adaptive Attacks to Adversarial Example Defenses

Abstract

Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS---and chosen for illustrative and pedagogical purposes---can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result---showing that a defense was ineffective---this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.

arXiv commentNeurIPS 2020

References in corpus (24)

• Auto-Encoding Variational Bayes
• Ensemble Adversarial Training: Attacks and Defenses
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• On Evaluating Adversarial Robustness
• Matrix estimation by Universal Singular Value Thresholding
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Defensive Distillation is Not Robust to Adversarial Examples
• Towards the first adversarially robust neural network model on MNIST
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
• Characterizing Audio Adversarial Examples Using Temporal Dependency
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Accurate, reliable and fast robustness evaluation
• Adversarial Robustness as a Prior for Learned Representations
• A New Defense Against Adversarial Images: Turning a Weakness into a Strength
• An Alternative Surrogate Loss for PGD-based Adversarial Testing
• Max-Mahalanobis Linear Discriminant Analysis Networks
• Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?
• Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
• Are Odds Really Odd? Bypassing Statistical Detection of Adversarial Examples
• EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attacks

Cited by in corpus (57)

• DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensembles
• Adversarial Distributional Training for Robust Deep Learning
• Bugs in our Pockets: The Risks of Client-Side Scanning
• Efficient Exact Verification of Binarized Neural Networks
• Towards Understanding Fast Adversarial Training
• RP2K: A Large-Scale Retail Product Dataset for Fine-Grained Image Classification
• Randomization matters. How to defend against strong adversarial attacks
• FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques
• Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques
• Geometry-Inspired Top-k Adversarial Perturbations
• SoK: Machine Learning Governance
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• Adversarial Robustness by Design through Analog Computing and Synthetic Gradients
• Security and Machine Learning in the Real World
• Optimal Transport as a Defense Against Adversarial Attacks
• Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attacks
• SOAR: Second-Order Adversarial Regularization
• A Partial Break of the Honeypots Defense to Catch Adversarial Attacks
• Adversarial Robustness of Supervised Sparse Coding
• Data-Dependent Randomized Smoothing
• Enhancing Certified Robustness via Smoothed Weighted Ensembling
• Automated Discovery of Adaptive Attacks on Adversarial Defenses
• BUZz: BUffer Zones for defending adversarial examples in image classification
• Improving Robustness using Generated Data
• Provably Robust Detection of Out-of-distribution Data (almost) for free
• Adversarially Optimized Mixup for Robust Classification
• Beware the Black-Box: on the Robustness of Recent Defenses to Adversarial Examples
• A Markov Game Model for AI-based Cyber Security Attack Mitigation
• Improve Adversarial Robustness via Weight Penalization on Classification Layer
• Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks
• Ptolemy: Architecture Support for Robust Deep Learning
• Voting for the right answer: Adversarial defense for speaker verification
• DetectorGuard: Provably Securing Object Detectors against Localized Patch Hiding Attacks
• Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• Understanding the Intrinsic Robustness of Image Distributions using Conditional Generative Models
• On the human-recognizability phenomenon of adversarially trained deep image classifiers
• Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training
• Bio-inspired Robustness: A Review
• Content-Adaptive Pixel Discretization to Improve Model Robustness
• On ${\ell }_p$-norm Robustness of Ensemble Stumps and Trees
• Who is Responsible for Adversarial Defense?
• Adversarial robustness via stochastic regularization of neural activation sensitivity
• A Survey on Threat Situation Awareness Systems: Framework, Techniques, and Insights
• Bridged Adversarial Training
• Delving into the pixels of adversarial samples
• Robustifying ${\ell }_{\infty }$ Adversarial Training to the Union of Perturbation Models
• Generalized Likelihood Ratio Test for Adversarially Robust Hypothesis Testing
• Adversarial Robustness Across Representation Spaces
• A Neuro-Inspired Autoencoding Defense Against Adversarial Perturbations
• Target Training Does Adversarial Training Without Adversarial Samples
• Adversarial Evaluation of Multimodal Models under Realistic Gray Box Assumption
• Dynamic Metric Learning: Towards a Scalable Metric Space to Accommodate Multiple Semantic Scales
• Tricking Adversarial Attacks To Fail
• Adversarial Examples and Metrics
• Improving Resistance to Adversarial Deformations by Regularizing Gradients
• Ensemble of Models Trained by Key-based Transformed Images for Adversarially Robust Defense Against Black-box Attacks