Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness

Abstract

Adversarial examples are malicious inputs crafted to cause a model to misclassify them. Their most common instantiation, "perturbation-based" adversarial examples introduce changes to the input that leave its true label unchanged, yet result in a different model prediction. Conversely, "invariance-based" adversarial examples insert changes to the input that leave the model's prediction unaffected despite the underlying input's label having changed. In this paper, we demonstrate that robustness to perturbation-based adversarial examples is not only insufficient for general robustness, but worse, it can also increase vulnerability of the model to invariance-based adversarial examples. In addition to analytical constructions, we empirically study vision classifiers with state-of-the-art robustness to perturbation-based adversaries constrained by an ${\ell }_p$ norm. We mount attacks that exploit excessive model invariance in directions relevant to the task, which are able to find adversarial examples within the ${\ell }_p$ ball. In fact, we find that classifiers trained to be ${\ell }_p$-norm robust are more vulnerable to invariance-based adversarial examples than their undefended counterparts. Excessive invariance is not limited to models trained to be robust to perturbation-based ${\ell }_p$-norm adversaries. In fact, we argue that the term adversarial example is used to capture a series of model limitations, some of which may not have been discovered yet. Accordingly, we call for a set of precise definitions that taxonomize and address each of these shortcomings in learning.

Accepted at the ICLR 2019 SafeML Workshop

References in corpus (9)

• Conditional Generative Adversarial Nets
• NICE: Non-linear Independent Components Estimation
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• On Evaluating Adversarial Robustness
• Towards the first adversarially robust neural network model on MNIST
• Motivating the Rules of the Game for Adversarial Example Research
• The Robust Manifold Defense: Adversarial Training using Generative Models
• Analysis of Invariance and Robustness via Invertibility of ReLU-Networks

Cited by in corpus (13)

• On Adaptive Attacks to Adversarial Example Defenses
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
• Testing Robustness Against Unforeseen Adversaries
• Batch Normalization is a Cause of Adversarial Vulnerability
• Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
• Towards an Adversarially Robust Normalization Approach
• Improving Certified Robustness via Statistical Learning with Logical Reasoning
• Rearchitecting Classification Frameworks For Increased Robustness
• Towards the Quantification of Safety Risks in Deep Neural Networks
• Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey
• DiPSeN: Differentially Private Self-normalizing Neural Networks For Adversarial Robustness in Federated Learning
• Discovering Invariances in Healthcare Neural Networks