RobustBench: a standardized adversarial robustness benchmark

Abstract

As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in training robust models. A key challenge in benchmarking robustness is that its evaluation is often error-prone leading to robustness overestimation. Our goal is to establish a standardized benchmark of adversarial robustness, which as accurately as possible reflects the robustness of the considered models within a reasonable computational budget. To this end, we start by considering the image classification task and introduce restrictions (possibly loosened in the future) on the allowed models. We evaluate adversarial robustness with AutoAttack, an ensemble of white- and black-box attacks, which was recently shown in a large-scale study to improve almost all robustness evaluations compared to the original publications. To prevent overadaptation of new defenses to AutoAttack, we welcome external evaluations based on adaptive attacks, especially where AutoAttack flags a potential overestimation of robustness. Our leaderboard, hosted at https://robustbench.github.io/, contains evaluations of 120+ models and aims at reflecting the current state of the art in image classification on a set of well-defined tasks in ${\ell }_{\infty }$- and ${\ell }_2$-threat models and on common corruptions, with possible extensions in the future. Additionally, we open-source the library https://github.com/RobustBench/robustbench that provides unified access to 80+ robust models to facilitate their downstream applications. Finally, based on the collected models, we analyze the impact of robustness on the performance on distribution shifts, calibration, out-of-distribution detection, fairness, privacy leakage, smoothness, and transferability.

arXiv commentThe camera-ready version accepted at the NeurIPS'21 Datasets and Benchmarks Track: 120+ evaluations, 80+ models, 7 leaderboards (Linf, L2, common corruptions; CIFAR-10, CIFAR-100, ImageNet), significantly expanded analysis part (calibration, fairness, privacy leakage, smoothness, transferability)

References in corpus (63)

• YOLOv4: Optimal Speed and Accuracy of Object Detection
• Intriguing properties of neural networks
• Wide Residual Networks
• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Certified Adversarial Robustness via Randomized Smoothing
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• On Evaluating Adversarial Robustness
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Fast is better than free: Revisiting adversarial training
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Robustness May Be at Odds with Accuracy
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• FreeLB: Enhanced Adversarial Training for Natural Language Understanding
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Exploring the Landscape of Spatial Robustness
• Adversarial Robustness through Local Linearization
• Measuring Robustness to Natural Distribution Shifts in Image Classification
• Motivating the Rules of the Game for Adversarial Example Research
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Adversarial Patch
• Adversarial Training and Robustness for Multiple Perturbations
• Fixing Data Augmentation to Improve Adversarial Robustness
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Adversarial Examples for Semantic Segmentation and Object Detection
• MMA Training: Direct Input Space Margin Maximization through Adversarial Training
• Functional Adversarial Attacks
• Evaluating and Understanding the Robustness of Adversarial Logit Pairing
• Testing Robustness Against Unforeseen Adversaries
• DeepRobust: A PyTorch Library for Adversarial Attacks and Defenses
• Adversarial Robustness Against the Union of Multiple Perturbation Models
• Unrestricted Adversarial Examples
• ME-Net: Towards Effective Adversarial Robustness with Matrix Estimation
• Metric Learning for Adversarial Robustness
• An Alternative Surrogate Loss for PGD-based Adversarial Testing
• Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
• Fawkes: Protecting Privacy against Unauthorized Deep Learning Models
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Learning perturbation sets for robust machine learning
• Advbox: a toolbox to generate adversarial examples that fool neural networks
• Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?
• Quantifying Perceptual Distortion of Adversarial Examples
• Transfer of Adversarial Robustness Between Perturbation Types
• To be Robust or to be Fair: Towards Fairness in Adversarial Training
• Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracy
• The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
• Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
• On the effectiveness of adversarial training against common corruptions
• Improving the Speed and Quality of GAN by Adversarial Training
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• A Critical Evaluation of Open-World Machine Learning
• Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attacks
• A critique of the DeepSec Platform for Security Analysis of Deep Learning Models
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• RoBIC: A benchmark suite for assessing classifiers robustness

Cited by in corpus (37)

• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Fixing Data Augmentation to Improve Adversarial Robustness
• Unsolved Problems in ML Safety
• RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
• Adversarial purification with Score-based generative models
• Sparse Visual Counterfactual Explanations in Image Space
• Test-time Batch Statistics Calibration for Covariate Shift
• What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
• On the effectiveness of adversarial training against common corruptions
• Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints
• An Evolutionary, Gradient-Free, Query-Efficient, Black-Box Algorithm for Generating Adversarial Instances in Deep Networks
• Attacking Adversarial Attacks as A Defense
• Protein Folding Neural Networks Are Not Robust
• Test time Adaptation through Perturbation Robustness
• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attacks
• Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness
• TSS: Transformation-Specific Smoothing for Robustness Certification
• Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?
• Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples
• Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselines
• Automated Discovery of Adaptive Attacks on Adversarial Defenses
• Mind the box: $l_1$-APGD for sparse adversarial attacks on image classifiers
• Improving Robustness using Generated Data
• A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• Defending Against Image Corruptions Through Adversarial Augmentations
• Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks
• RoBIC: A benchmark suite for assessing classifiers robustness
• Understanding and Achieving Efficient Robustness with Adversarial Supervised Contrastive Learning
• Lower Bounds on Cross-Entropy Loss in the Presence of Test-time Adversaries
• Bio-inspired Robustness: A Review
• A Discussion on Building Practical NLP Leaderboards: The Case of Machine Translation
• Training Efficiency and Robustness in Deep Learning
• Automated Decision-based Adversarial Attacks
• Subspace Adversarial Training