Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability

Abstract

We explore the concept of co-design in the context of neural network verification. Specifically, we aim to train deep neural networks that not only are robust to adversarial perturbations but also whose robustness can be verified more easily. To this end, we identify two properties of network models - weight sparsity and so-called ReLU stability - that turn out to significantly impact the complexity of the corresponding verification task. We demonstrate that improving weight sparsity alone already enables us to turn computationally intractable verification problems into tractable ones. Then, improving ReLU stability leads to an additional 4-13x speedup in verification times. An important feature of our methodology is its "universality," in the sense that it can be used with a broad range of training procedures and verification approaches.

References in corpus (8)

• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Defenses against Adversarial Examples
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• An approach to reachability analysis for feed-forward ReLU neural networks
• Training verified learners with learned verifiers

Cited by in corpus (40)

• Adversarial Examples Are Not Bugs, They Are Features
• Robustness May Be at Odds with Accuracy
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Verification of Neural Network Behaviour: Formal Guarantees for Power System Applications
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack
• Overfitting in adversarially robust deep learning
• The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Towards Stable and Efficient Training of Verifiably Robust Neural Networks
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• Efficient Exact Verification of Binarized Neural Networks
• Exactly Computing the Local Lipschitz Constant of ReLU Networks
• Lipschitz regularized Deep Neural Networks generalize and are adversarially robust
• Handcrafted Backdoors in Deep Neural Networks
• Breaking certified defenses: Semantic adversarial examples with spoofed robustness certificates
• The Search for Sparse, Robust Neural Networks
• Equivalent and Approximate Transformations of Deep Neural Networks
• Manifold Regularization for Locally Stable Deep Neural Networks
• Neural Networks for Encoding Dynamic Security-Constrained Optimal Power Flow
• Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
• Verification of Neural Networks: Enhancing Scalability through Pruning
• Fast Certified Robust Training with Short Warmup
• Boosting the Certified Robustness of L-infinity Distance Nets
• Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers
• Scaling Up Exact Neural Network Compression by ReLU Stability
• Provable robustness against all adversarial $l_p$-perturbations for $p\ge 1$
• Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons
• Fast and Stable Interval Bounds Propagation for Training Verifiably Robust Models
• Adversarial Robustness Guarantees for Random Deep Neural Networks
• Fast Geometric Projections for Local Robustness Certification
• Universal Approximation with Certified Networks
• A Generative Model based Adversarial Security of Deep Learning and Linear Classifier Models
• Blind Adversarial Pruning: Balance Accuracy, Efficiency and Robustness
• Query complexity of adversarial attacks
• Resilience from Diversity: Population-based approach to harden models against adversarial attacks
• Adaptive Image Transformations for Transfer-based Adversarial Attack
• Training Provably Robust Models by Polyhedral Envelope Regularization
• Gödel's Sentence Is An Adversarial Example But Unsolvable