Manifold Regularization for Locally Stable Deep Neural Networks

Abstract

We apply concepts from manifold regularization to develop new regularization techniques for training locally stable deep neural networks. Our regularizers are based on a sparsification of the graph Laplacian which holds with high probability when the data is sparse in high dimensions, as is common in deep learning. Empirically, our networks exhibit stability in a diverse set of perturbation models, including ${\ell }_2$, ${\ell }_{\infty }$, and Wasserstein-based perturbations; in particular, we achieve 40% adversarial accuracy on CIFAR-10 against an adaptive PGD attack using ${\ell }_{\infty }$ perturbations of size $\epsilon =8/255$, and state-of-the-art verified accuracy of 21% in the same perturbation model. Furthermore, our techniques are efficient, incurring overhead on par with two additional parallel forward passes through the network.

References in corpus (18)

• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• On Evaluating Adversarial Robustness
• Consistency of spectral clustering
• Fast is better than free: Revisiting adversarial training
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Are Labels Required for Improving Adversarial Robustness?
• A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
• Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
• Adversarial Robustness Against the Union of Multiple Perturbation Models
• Manifold Regularized Deep Neural Networks using Adversarial Examples
• Graph based manifold regularized deep neural networks for automatic speech recognition

Cited by in corpus (4)

• How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses
• Recent Advances in Understanding Adversarial Robustness of Deep Neural Networks
• LAFEAT: Piercing Through Adversarial Defenses with Latent Features
• Deep Repulsive Prototypes for Adversarial Robustness