Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond

Abstract

Linear relaxation based perturbation analysis (LiRPA) for neural networks, which computes provable linear bounds of output neurons given a certain amount of input perturbation, has become a core component in robustness verification and certified defense. The majority of LiRPA-based methods focus on simple feed-forward networks and need particular manual derivations and implementations when extended to other architectures. In this paper, we develop an automatic framework to enable perturbation analysis on any neural network structures, by generalizing existing LiRPA algorithms such as CROWN to operate on general computational graphs. The flexibility, differentiability and ease of use of our framework allow us to obtain state-of-the-art results on LiRPA based certified defense on fairly complicated networks like DenseNet, ResNeXt and Transformer that are not supported by prior works. Our framework also enables loss fusion, a technique that significantly reduces the computational complexity of LiRPA for certified defense. For the first time, we demonstrate LiRPA based certified defense on Tiny ImageNet and Downscaled ImageNet where previous approaches cannot scale to due to the relatively large number of classes. Our work also yields an open-source library for the community to apply LiRPA to areas beyond certified defense without much LiRPA expertise, e.g., we create a neural network with a probably flat optimization landscape by applying LiRPA to network parameters. Our opensource library is available at https://github.com/KaidiXu/auto_LiRPA.

References in corpus (23)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• Efficient Neural Network Robustness Certification with General Activation Functions
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Semidefinite relaxations for certifying robustness to adversarial examples
• Certified Adversarial Robustness with Additive Noise
• Efficient Formal Safety Analysis of Neural Networks
• A Dual Approach to Scalable Verification of Deep Networks
• A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
• Training verified learners with learned verifiers
• POPQORN: Quantifying Robustness of Recurrent Neural Networks
• The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification
• Asymmetric Valleys: Beyond Sharp and Flat Local Minima
• Robustness Verification for Transformers
• Enhancing Certifiable Robustness via a Deep Model Ensemble
• Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers

Cited by in corpus (7)

• PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximations
• Certification of embedded systems based on Machine Learning: A survey
• TSS: Transformation-Specific Smoothing for Robustness Certification
• Contraction-Guided Adaptive Partitioning for Reachability Analysis of Neural Network Controlled Systems
• Verification of Neural Network Control Systems in Continuous Time
• Towards Evaluating and Training Verifiably Robust Neural Networks
• Towards Robustness Analysis of E-Commerce Ranking System