Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness

Abstract

Evaluating the robustness of a defense model is a challenging task in adversarial robustness research. Obfuscated gradients have previously been found to exist in many defense methods and cause a false signal of robustness. In this paper, we identify a more subtle situation called Imbalanced Gradients that can also cause overestimated adversarial robustness. The phenomenon of imbalanced gradients occurs when the gradient of one term of the margin loss dominates and pushes the attack towards to a suboptimal direction. To exploit imbalanced gradients, we formulate a Margin Decomposition (MD) attack that decomposes a margin loss into individual terms and then explores the attackability of these terms separately via a two-stage process. We also propose a multi-targeted and ensemble version of our MD attack. By investigating 24 defense models proposed since 2018, we find that 11 models are susceptible to a certain degree of imbalanced gradients and our MD attack can decrease their robustness evaluated by the best standalone baseline attack by more than 1%. We also provide an in-depth investigation on the likely causes of imbalanced gradients and effective countermeasures. Our code is available at https://github.com/HanxunH/MDAttack.

arXiv commentTo appear in Machine Learning

References in corpus (25)

• Explaining and Harnessing Adversarial Examples
• Ensemble Adversarial Training: Attacks and Defenses
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Fast is better than free: Revisiting adversarial training
• Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
• Countering Adversarial Images using Input Transformations
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Adversarial Weight Perturbation Helps Robust Generalization
• Are Labels Required for Improving Adversarial Robustness?
• Adversarially Robust Distillation
• Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• RobustBench: a standardized adversarial robustness benchmark
• Unlabeled Data Improves Adversarial Robustness
• Boosting Adversarial Training with Hypersphere Embedding
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• Diversity can be Transferred: Output Diversification for White- and Black-box Attacks
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
• Jacobian Adversarially Regularized Networks for Robustness
• Learning Diverse-Structured Networks for Adversarial Robustness