Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations

Abstract

Adversarial examples are malicious inputs crafted to induce misclassification. Commonly studied sensitivity-based adversarial examples introduce semantically-small changes to an input that result in a different model prediction. This paper studies a complementary failure mode, invariance-based adversarial examples, that introduce minimal semantic changes that modify an input's true label yet preserve the model's prediction. We demonstrate fundamental tradeoffs between these two types of adversarial examples. We show that defenses against sensitivity-based attacks actively harm a model's accuracy on invariance-based attacks, and that new approaches are needed to resist both attack types. In particular, we break state-of-the-art adversarially-trained and certifiably-robust models by generating small perturbations that the models are (provably) robust to, yet that change an input's class according to human labelers. Finally, we formally show that the existence of excessively invariant classifiers arises from the presence of overly-robust predictive features in standard datasets.

arXiv commentICML 2020 (Supersedes the workshop paper "Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness", arXiv:1903.10484)

References in corpus (15)

• Conditional Generative Adversarial Nets
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Defenses against Adversarial Examples
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Exploring the Landscape of Spatial Robustness
• Towards the first adversarially robust neural network model on MNIST
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• Motivating the Rules of the Game for Adversarial Example Research
• The Robust Manifold Defense: Adversarial Training using Generative Models
• A Fourier Perspective on Model Robustness in Computer Vision
• Testing Robustness Against Unforeseen Adversaries
• Adversarial Robustness as a Prior for Learned Representations
• Discretization based Solutions for Secure Machine Learning against Adversarial Attacks
• Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
• Towards Deep Learning Models Resistant to Large Perturbations

Cited by in corpus (15)

• Blind Backdoors in Deep Learning Models
• Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
• Bugs in our Pockets: The Risks of Client-Side Scanning
• Exposing Previously Undetectable Faults in Deep Neural Networks
• To be Robust or to be Fair: Towards Fairness in Adversarial Training
• Hold me tight! Influence of discriminative features on deep network boundaries
• Evaluating Robustness to Context-Sensitive Feature Perturbations of Different Granularities
• Improving Robustness using Generated Data
• Invariance Measures for Neural Networks
• Models of Computational Profiles to Study the Likelihood of DNN Metamorphic Test Cases
• MixDefense: A Defense-in-Depth Framework for Adversarial Example Detection Based on Statistical and Semantic Analysis
• Can we have it all? On the Trade-off between Spatial and Adversarial Robustness of Neural Networks
• Understanding the Interplay between Privacy and Robustness in Federated Learning
• Balancing Robustness and Sensitivity using Feature Contrastive Learning
• Pareto Adversarial Robustness: Balancing Spatial Robustness and Sensitivity-based Robustness