Weight-Covariance Alignment for Adversarially Robust Neural Networks

Abstract

Stochastic Neural Networks (SNNs) that inject noise into their hidden layers have recently been shown to achieve strong robustness against adversarial attacks. However, existing SNNs are usually heuristically motivated, and often rely on adversarial training, which is computationally costly. We propose a new SNN that achieves state-of-the-art performance without relying on adversarial training, and enjoys solid theoretical justification. Specifically, while existing SNNs inject learned or hand-tuned isotropic noise, our SNN learns an anisotropic noise distribution to optimize a learning-theoretic bound on adversarial robustness. We evaluate our method on a number of popular benchmarks, show that it can be applied to different architectures, and that it provides robustness to a variety of white-box and black-box attacks, while being simple and fast to train compared to existing alternatives.

References in corpus (13)

• Auto-Encoding Variational Bayes
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Certified Adversarial Robustness via Randomized Smoothing
• Certified Adversarial Robustness with Additive Noise
• Hessian Schatten-Norm Regularization for Linear Inverse Problems
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
• Robust Local Features for Improving the Generalization of Adversarial Training
• Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers
• Theoretical evidence for adversarial robustness through randomization