Towards the first adversarially robust neural network model on MNIST

Abstract

Despite much effort, deep neural networks remain highly susceptible to tiny input perturbations and even for MNIST, one of the most common toy datasets in computer vision, no neural network model exists for which adversarial perturbations are large and make semantic sense to humans. We show that even the widely recognized and by far most successful defense by Madry et al. (1) overfits on the L-infinity metric (it's highly susceptible to L2 and L0 perturbations), (2) classifies unrecognizable images with high certainty, (3) performs not much better than simple input binarization and (4) features adversarial perturbations that make little sense to humans. These results suggest that MNIST is far from being solved in terms of adversarial robustness. We present a novel robust classification model that performs analysis by synthesis using learned class-conditional data distributions. We derive bounds on the robustness and go to great length to empirically evaluate our model using maximally effective adversarial attacks by (a) applying decision-based, score-based, gradient-based and transfer-based attacks for several different Lp norms, (b) by designing a new attack that exploits the structure of our defended model and (c) by devising a novel decision-based attack that seeks to minimize the number of perturbed pixels (L0). The results suggest that our approach yields state-of-the-art robustness on MNIST against L0, L2 and L-infinity perturbations and we demonstrate that most adversarial examples are strongly perturbed towards the perceptual boundary between the original and the adversarial class.

Cited by in corpus (78)

• Shortcut Learning in Deep Neural Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• On Evaluating Adversarial Robustness
• Invertible Residual Networks
• On Adaptive Attacks to Adversarial Example Defenses
• Adversarial Training and Robustness for Multiple Perturbations
• Causality for Machine Learning
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Going in circles is the way forward: the role of recurrence in visual inference
• Smooth Adversarial Training
• Accurate, reliable and fast robustness evaluation
• Adversarial Robustness Against the Union of Multiple Perturbation Models
• Excessive Invariance Causes Adversarial Vulnerability
• Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations
• Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
• Batch Normalization is a Cause of Adversarial Vulnerability
• Adversarial Examples in Modern Machine Learning: A Review
• RoPGen: Towards Robust Code Authorship Attribution via Automatic Coding Style Transformation
• Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
• Recent Advances in Adversarial Training for Adversarial Robustness
• A Causal View on Robustness of Neural Networks
• DARCCC: Detecting Adversaries by Reconstruction from Class Conditional Capsules
• Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
• Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing
• Adversarial purification with Score-based generative models
• Are Generative Classifiers More Robust to Adversarial Attacks?
• Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study
• GenAttack: Practical Black-box Attacks with Gradient-Free Optimization
• Nonlinear Invariant Risk Minimization: A Causal Approach
• Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
• Defending Against Adversarial Examples with K-Nearest Neighbor
• Physically Realizable Adversarial Examples for LiDAR Object Detection
• Biologically Inspired Mechanisms for Adversarial Robustness
• T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification
• Understanding the Limitations of Conditional Generative Models
• Adaptive Generation of Unrestricted Adversarial Inputs
• Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attacks
• Adversarial Visual Robustness by Causal Intervention
• ShapeAdv: Generating Shape-Aware Adversarial 3D Point Clouds
• Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithm
• Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification
• A Bayes-Optimal View on Adversarial Examples
• Towards Defending Multiple ${\ell }_p$-norm Bounded Adversarial Perturbations via Gated Batch Normalization
• Discovering Robust Convolutional Architecture at Targeted Capacity: A Multi-Shot Approach
• Towards Rapid and Robust Adversarial Training with One-Step Attacks
• Toward Adversarial Robustness via Semi-supervised Robust Training
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Mind the box: $l_1$-APGD for sparse adversarial attacks on image classifiers
• Affine Variational Autoencoders: An Efficient Approach for Improving Generalization and Robustness to Distribution Shift
• Statistically Robust Neural Network Classification
• Robust and Private Learning of Halfspaces
• Architecture Selection via the Trade-off Between Accuracy and Robustness
• Provable robustness against all adversarial $l_p$-perturbations for $p\ge 1$
• Towards a Theoretical Understanding of the Robustness of Variational Autoencoders
• Provable Robust Classification via Learned Smoothed Densities
• ROBY: Evaluating the Robustness of a Deep Model by its Decision Boundaries
• Adversarial Robustness Guarantees for Random Deep Neural Networks
• Score-Based Generative Classifiers
• Towards Deep Learning Models Resistant to Large Perturbations
• A Multiclass Boosting Framework for Achieving Fast and Provable Adversarial Robustness
• Mapper Based Classifier
• Empirically Measuring Concentration: Fundamental Limits on Intrinsic Robustness
• Provably robust deep generative models
• Understanding and Quantifying Adversarial Examples Existence in Linear Classification
• A cryptographic approach to black box adversarial machine learning
• Neural Belief Reasoner
• An Ensemble Approach Towards Adversarial Robustness
• Adversarial Attack with Pattern Replacement
• Learning to Disentangle Robust and Vulnerable Features for Adversarial Detection
• Learning to Separate Clusters of Adversarial Representations for Robust Adversarial Detection
• Utilizing a null class to restrict decision spaces and defend against neural network adversarial attacks
• Unsupervised Learning of Multi-level Structures for Anomaly Detection
• Multi-way Encoding for Robustness
• Targeted Physical-World Attention Attack on Deep Learning Models in Road Sign Recognition
• Adversarial Examples as an Input-Fault Tolerance Problem
• Robust Face Verification via Disentangled Representations
• Adversarial defenses via a mixture of generators
• Towards Optimal Randomized Strategies in Adversarial Example Game