Rethinking Empirical Evaluation of Adversarial Robustness Using First-Order Attack Methods
arXiv:2006.01304
Abstract
We identify three common cases that lead to overestimation of adversarial accuracy against bounded first-order attack methods, which is popularly used as a proxy for adversarial robustness in empirical studies. For each case, we propose compensation methods that either address sources of inaccurate gradient computation, such as numerical instability near zero and non-differentiability, or reduce the total number of back-propagations for iterative attacks by approximating second-order information. These compensation methods can be combined with existing attack methods for a more precise empirical evaluation metric. We illustrate the impact of these three cases with examples of practical interest, such as benchmarking model capacity and regularization techniques for robustness. Overall, our work shows that overestimated adversarial accuracy that is not indicative of robustness is prevalent even for conventionally trained deep neural networks, and highlights cautions of using empirical evaluation without guaranteed bounds.
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Intriguing properties of neural networks
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Evaluating Robustness of Neural Networks with Mixed Integer Programming
- Certifying Some Distributional Robustness with Principled Adversarial Training
- Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
- advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
- The Limitations of Deep Learning in Adversarial Settings
- Architecture Selection via the Trade-off Between Accuracy and Robustness