Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

Abstract

The field of defense strategies against adversarial attacks has significantly grown over the last years, but progress is hampered as the evaluation of adversarial defenses is often insufficient and thus gives a wrong impression of robustness. Many promising defenses could be broken later on, making it difficult to identify the state-of-the-art. Frequent pitfalls in the evaluation are improper tuning of hyperparameters of the attacks, gradient obfuscation or masking. In this paper we first propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness. We apply our ensemble to over 50 models from papers published at recent top machine learning and computer vision venues. In all except one of the cases we achieve lower robust test accuracy than reported in these papers, often by more than $10\%$, identifying several broken defenses.

In ICML 2020

References in corpus (19)

• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Certified Adversarial Robustness via Randomized Smoothing
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Fast is better than free: Revisiting adversarial training
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• Adversarial Robustness through Local Linearization
• Certified Adversarial Robustness with Additive Noise
• Are Labels Required for Improving Adversarial Robustness?
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• MMA Training: Direct Input Space Margin Maximization through Adversarial Training
• Improving the Generalization of Adversarial Training with Domain Adaptation
• Provable Robustness of ReLU networks via Maximization of Linear Regions
• Controlling Neural Level Sets
• Metric Learning for Adversarial Robustness
• Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models

Cited by in corpus (114)

• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Adversarial Weight Perturbation Helps Robust Generalization
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• On Adaptive Attacks to Adversarial Example Defenses
• Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
• RobustBench: a standardized adversarial robustness benchmark
• Fixing Data Augmentation to Improve Adversarial Robustness
• A Review of Adversarial Attack and Defense for Classification Methods
• Smooth Adversarial Training
• Adversarial Training against Location-Optimized Adversarial Patches
• Logic Explained Networks
• Boosting Adversarial Training with Hypersphere Embedding
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
• FedDef: Defense Against Gradient Leakage in Federated Learning-based Network Intrusion Detection Systems
• Understanding and Improving Fast Adversarial Training
• Semantic-Aware Adversarial Training for Reliable Deep Hashing Retrieval
• On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
• HYDRA: Pruning Adversarially Robust Neural Networks
• Geometry-aware Instance-reweighted Adversarial Training
• Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspective
• Adversarial Robustness via Fisher-Rao Regularization
• Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
• Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
• SAT: Improving Adversarial Training via Curriculum-Based Loss Smoothing
• On the effectiveness of adversarial training against common corruptions
• InfoAT: Improving Adversarial Training Using the Information Bottleneck Principle
• Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints
• Exploring Memorization in Adversarial Training
• Bag of Tricks for Adversarial Training
• Adversarial Example Games
• Learnable Boundary Guided Adversarial Training
• On Improving Adversarial Transferability of Vision Transformers
• CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection
• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training
• Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Manifold Regularization for Locally Stable Deep Neural Networks
• Label Smoothing and Adversarial Robustness
• Adversarial Robustness under Long-Tailed Distribution
• Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models
• Bit Error Robustness for Energy-Efficient DNN Accelerators
• Dynamically Sampled Nonlocal Gradients for Stronger Adversarial Attacks
• Probing the Purview of Neural Networks via Gradient Analysis
• Adversarial Robustness through the Lens of Convolutional Filters
• TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness
• Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning
• Robust Proxy: Improving Adversarial Robustness by Robust Proxy Learning
• Boosting the Certified Robustness of L-infinity Distance Nets
• SOAR: Second-Order Adversarial Regularization
• Adversarial Attack and Defense in Deep Ranking
• Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?
• RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation
• Robust Models Are More Interpretable Because Attributions Look Normal
• DiffDefense: Defending against Adversarial Attacks via Diffusion Models
• Enhancing Certified Robustness via Smoothed Weighted Ensembling
• Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples
• Adversarial Detection: Attacking Object Detection in Real Time
• Towards Defending Multiple ${\ell }_p$-norm Bounded Adversarial Perturbations via Gated Batch Normalization
• Provably Robust Detection of Out-of-distribution Data (almost) for free
• Data Quality Matters For Adversarial Training: An Empirical Study
• Improving Robustness using Generated Data
• Mind the box: $l_1$-APGD for sparse adversarial attacks on image classifiers
• Towards Evaluating the Robustness of Neural Networks Learned by Transduction
• DeepNNK: Explaining deep models and their generalization using polytope interpolation
• Automated Discovery of Adaptive Attacks on Adversarial Defenses
• Practical Relative Order Attack in Deep Ranking
• Medical Aegis: Robust adversarial protectors for medical images
• Learning Diverse-Structured Networks for Adversarial Robustness
• Adversarial Attack across Datasets
• Improving Model Robustness with Latent Distribution Locally and Globally
• Understanding Catastrophic Overfitting in Adversarial Training
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks
• Push Stricter to Decide Better: A Class-Conditional Feature Adaptive Framework for Improving Adversarial Robustness
• Clustering Effect of (Linearized) Adversarial Robust Models
• Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks
• Smoothness Analysis of Adversarial Training
• Understanding and Achieving Efficient Robustness with Adversarial Supervised Contrastive Learning
• Iterative Window Mean Filter: Thwarting Diffusion-based Adversarial Purification
• Adversarial Feature Desensitization
• ZeroGrad : Mitigating and Explaining Catastrophic Overfitting in FGSM Adversarial Training
• Adaptive Image Transformations for Transfer-based Adversarial Attack
• Real World Robustness from Systematic Noise
• Lower Bounds on Cross-Entropy Loss in the Presence of Test-time Adversaries
• Improving Fast Minimum-Norm Attacks with Hyperparameter Optimization
• ImageNet Pre-training also Transfers Non-Robustness
• Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Training
• Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training
• Towards Evaluating and Training Verifiably Robust Neural Networks
• Domain Invariant Adversarial Learning
• A Perceptual Distortion Reduction Framework: Towards Generating Adversarial Examples with High Perceptual Quality and Attack Success Rate
• Removing Undesirable Feature Contributions Using Out-of-Distribution Data
• Enhance Diffusion to Improve Robust Generalization
• Tightening the Approximation Error of Adversarial Risk with Auto Loss Function Search
• Training Provably Robust Models by Polyhedral Envelope Regularization
• Efficient and Transferable Adversarial Examples from Bayesian Neural Networks
• Introducing the DOME Activation Functions
• Improving Resistance to Adversarial Deformations by Regularizing Gradients
• Efficient Adversarial Input Generation via Neural Net Patching
• BAARD: Blocking Adversarial Examples by Testing for Applicability, Reliability and Decidability
• Bridged Adversarial Training
• Towards Optimal Randomized Strategies in Adversarial Example Game
• Harnessing Perceptual Adversarial Patches for Crowd Counting
• Adversarial attacks on neural networks through canonical Riemannian foliations
• FROB: Few-shot ROBust Model for Classification and Out-of-Distribution Detection
• Subspace Adversarial Training
• Amicable Aid: Perturbing Images to Improve Classification Performance
• Dual Head Adversarial Training