Excessive Invariance Causes Adversarial Vulnerability

Abstract

Despite their impressive performance, deep neural networks exhibit striking failures on out-of-distribution inputs. One core idea of adversarial example research is to reveal neural network errors under such distribution shifts. We decompose these errors into two complementary sources: sensitivity and invariance. We show deep networks are not only too sensitive to task-irrelevant changes of their input, as is well-known from epsilon-adversarial examples, but are also too invariant to a wide range of task-relevant changes, thus making vast regions in input space vulnerable to adversarial attacks. We show such excessive invariance occurs across various tasks and architecture types. On MNIST and ImageNet one can manipulate the class-specific content of almost any image without changing the hidden activations. We identify an insufficiency of the standard cross-entropy loss as a reason for these failures. Further, we extend this objective based on an information-theoretic analysis so it encourages the model to consider all task-dependent features in its decision. This provides the first approach tailored explicitly to overcome excessive invariance and resulting vulnerabilities.

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Estimating Mutual Information
• Learning deep representations by mutual information estimation and maximization
• Opening the Black Box of Deep Neural Networks via Information
• Understanding Deep Convolutional Networks
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• Deep Variational Information Bottleneck
• Towards the first adversarially robust neural network model on MNIST
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• Invertible Residual Networks
• Motivating the Rules of the Game for Adversarial Example Research
• Adversarial vulnerability for any classifier
• Analyzing Inverse Problems with Invertible Neural Networks
• Adversarial examples from computational constraints
• Unrestricted Adversarial Examples
• Analysis of Invariance and Robustness via Invertibility of ReLU-Networks
• Interaction Information for Causal Inference: The Case of Directed Triangle

Cited by in corpus (23)

• Shortcut Learning in Deep Neural Networks
• Guided Image Generation with Conditional Invertible Neural Networks
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
• DIVA: Domain Invariant Variational Autoencoders
• Benchmarking Attribution Methods with Relative Feature Importance
• Gradient Starvation: A Learning Proclivity in Neural Networks
• Towards an Adversarially Robust Normalization Approach
• Training Invertible Neural Networks as Autoencoders
• Invertible DenseNets with Concatenated LipSwish
• Do humans and Convolutional Neural Networks attend to similar areas during scene classification: Effects of task and image type
• Dangers of Bayesian Model Averaging under Covariate Shift
• Rearchitecting Classification Frameworks For Increased Robustness
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• DiPSeN: Differentially Private Self-normalizing Neural Networks For Adversarial Robustness in Federated Learning
• Learning to Find Correlated Features by Maximizing Information Flow in Convolutional Neural Networks
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation
• Exposing Semantic Segmentation Failures via Maximum Discrepancy Competition
• The Effects of Image Distribution and Task on Adversarial Robustness
• Guess First to Enable Better Compression and Adversarial Robustness
• Conditional Invertible Neural Networks for Diverse Image-to-Image Translation