Globally-Robust Neural Networks

Abstract

The threat of adversarial examples has motivated work on training certifiably robust neural networks to facilitate efficient verification of local robustness at inference time. We formalize a notion of global robustness, which captures the operational properties of on-line local robustness certification while yielding a natural learning objective for robust training. We show that widely-used architectures can be easily adapted to this objective by incorporating efficient global Lipschitz bounds into the network, yielding certifiably-robust models by construction that achieve state-of-the-art verifiable accuracy. Notably, this approach requires significantly less time and memory than recent certifiable training methods, and leads to negligible costs when certifying points on-line; for example, our evaluation shows that it is possible to train a large robust Tiny-Imagenet model in a matter of hours. Our models effectively leverage inexpensive global Lipschitz bounds for real-time certification, despite prior suggestions that tighter local bounds are needed for good performance; we posit this is possible because our models are specifically trained to achieve tighter global bounds. Namely, we prove that the maximum achievable verifiable accuracy for a given dataset is not improved by using a local bound.

arXiv commentAppearing in ICML 2021

References in corpus (19)

• Spectral Normalization for Generative Adversarial Networks
• Theoretically Principled Trade-off between Robustness and Accuracy
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• Certified Defenses against Adversarial Examples
• Efficient Neural Network Robustness Certification with General Activation Functions
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Adversarial Robustness through Local Linearization
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• The Singular Values of Convolutional Layers
• Generalizable Adversarial Training via Spectral Normalization
• MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
• Provable Robustness of ReLU networks via Maximization of Linear Regions
• Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks
• Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networks
• Provable Certificates for Adversarial Examples: Fitting a Ball in the Union of Polytopes
• Orthogonalizing Convolutional Layers with the Cayley Transform
• Universal Lipschitz Approximation in Bounded Depth Neural Networks

Cited by in corpus (4)

• Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective
• Learning Security Classifiers with Verified Global Robustness Properties
• Architecture-Preserving Provable Repair of Deep Neural Networks
• Boosting the Certified Robustness of L-infinity Distance Nets