Scaling provable adversarial defenses

Abstract

Recent work has developed methods for learning deep network classifiers that are provably robust to norm-bounded adversarial perturbation; however, these methods are currently only possible for relatively small feedforward networks. In this paper, in an effort to scale these approaches to substantially larger models, we extend previous work in three main directions. First, we present a technique for extending these training procedures to much more general networks, with skip connections (such as ResNets) and general nonlinearities; the approach is fully modular, and can be implemented automatically (analogous to automatic differentiation). Second, in the specific case of ${\ell }_{\infty }$ adversarial perturbations and networks with ReLU nonlinearities, we adopt a nonlinear random projection for training, which scales linearly in the number of hidden units (previous approaches scaled quadratically). Third, we show how to further improve robust error through cascade models. On both MNIST and CIFAR data sets, we train classifiers that improve substantially on the state of the art in provable robust adversarial error bounds: from 5.8% to 3.1% on MNIST (with ${\ell }_{\infty }$ perturbations of $\epsilon =0.1$), and from 80% to 36.4% on CIFAR (with ${\ell }_{\infty }$ perturbations of $\epsilon =2/255$). Code for all experiments in the paper is available at https://github.com/locuslab/convex_adversarial/.

References in corpus (7)

• Wide Residual Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Adversarial Machine Learning at Scale
• Certifying Some Distributional Robustness with Principled Adversarial Training
• An approach to reachability analysis for feed-forward ReLU neural networks
• A Dual Approach to Scalable Verification of Deep Networks

Cited by in corpus (14)

• Towards quantum enhanced adversarial robustness in machine learning
• PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximations
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Adversarial Robustness via Fisher-Rao Regularization
• Learning Security Classifiers with Verified Global Robustness Properties
• Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy
• Towards Understanding Fast Adversarial Training
• Adversarial Robustness by Design through Analog Computing and Synthetic Gradients
• On Certifying Non-uniform Bound against Adversarial Attacks
• Universal Lipschitz Approximation in Bounded Depth Neural Networks
• Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
• Augmenting Model Robustness with Transformation-Invariant Attacks
• Domain Invariant Adversarial Learning
• Weighted Average Precision: Adversarial Example Detection in the Visual Perception of Autonomous Vehicles