An Alternative Surrogate Loss for PGD-based Adversarial Testing

Abstract

Adversarial testing methods based on Projected Gradient Descent (PGD) are widely used for searching norm-bounded perturbations that cause the inputs of neural networks to be misclassified. This paper takes a deeper look at these methods and explains the effect of different hyperparameters (i.e., optimizer, step size and surrogate loss). We introduce the concept of MultiTargeted testing, which makes clever use of alternative surrogate losses, and explain when and how MultiTargeted is guaranteed to find optimal perturbations. Finally, we demonstrate that MultiTargeted outperforms more sophisticated methods and often requires less iterative steps than other variants of PGD found in the literature. Notably, MultiTargeted ranks first on MadryLab's white-box MNIST and CIFAR-10 leaderboards, reducing the accuracy of their MNIST model to 88.36% (with ${\ell }_{\infty }$ perturbations of $\epsilon =0.3$) and the accuracy of their CIFAR-10 model to 44.03% (at $\epsilon =8/255$). MultiTargeted also ranks first on the TRADES leaderboard reducing the accuracy of their CIFAR-10 model to 53.07% (with ${\ell }_{\infty }$ perturbations of $\epsilon =0.031$).

References in corpus (9)

• Explaining and Harnessing Adversarial Examples
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• On Evaluating Adversarial Robustness
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Adversarial Robustness through Local Linearization
• Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
• Feature Denoising for Improving Adversarial Robustness
• Enhancing Gradient-based Attacks with Symbolic Intervals

Cited by in corpus (27)

• Are Labels Required for Improving Adversarial Robustness?
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• On Adaptive Attacks to Adversarial Example Defenses
• RobustBench: a standardized adversarial robustness benchmark
• Fixing Data Augmentation to Improve Adversarial Robustness
• A Closer Look at Accuracy vs. Robustness
• Boosting Adversarial Training with Hypersphere Embedding
• Diversity can be Transferred: Output Diversification for White- and Black-box Attacks
• Understanding and Improving Fast Adversarial Training
• Adversarial Examples Make Strong Poisons
• Attacking Adversarial Attacks as A Defense
• Label Smoothing and Adversarial Robustness
• Abstract Interpretation of Fixpoint Iterators with Applications to Neural Networks
• Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks
• Improving Robustness using Generated Data
• Robust and Private Learning of Halfspaces
• LAFEAT: Piercing Through Adversarial Defenses with Latent Features
• Toward Few-step Adversarial Training from a Frequency Perspective
• Towards Deep Learning Models Resistant to Large Perturbations
• Smoothed Inference for Adversarially-Trained Models
• Improving Hierarchical Adversarial Robustness of Deep Neural Networks
• Adversarial Attacks on ML Defense Models Competition
• A Multiclass Boosting Framework for Achieving Fast and Provable Adversarial Robustness
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness
• Initializing Perturbations in Multiple Directions for Fast Adversarial Training
• A Closer Look at the Adversarial Robustness of Information Bottleneck Models
• Towards Sharper First-Order Adversary with Quantized Gradients