Provable Robustness of ReLU networks via Maximization of Linear Regions

Abstract

It has been shown that neural network classifiers are not robust. This raises concerns about their usage in safety-critical systems. We propose in this paper a regularization scheme for ReLU networks which provably improves the robustness of the classifier by maximizing the linear regions of the classifier as well as the distance to the decision boundary. Our techniques allow even to find the minimal adversarial perturbation for a fraction of test points for large networks. In the experiments we show that our approach improves upon adversarial training both in terms of lower and upper bounds on the robustness and is comparable or better than the state-of-the-art in terms of test error and robustness.

arXiv commentIn AISTATS 2019. Conference version with the following modifications: improved readability, comparison to Xiao et al (2018) added, section on visualizations extended

Cited by in corpus (36)

• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Fast is better than free: Revisiting adversarial training
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers
• On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systems
• Denoised Smoothing: A Provable Defense for Pretrained Classifiers
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
• Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing
• Randomized Smoothing of All Shapes and Sizes
• Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks
• Globally-Robust Neural Networks
• Adversarial Robustness via Fisher-Rao Regularization
• Smoothed Geometry for Robust Attribution
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Towards Robust, Locally Linear Deep Networks
• Understanding Adversarial Robustness: The Trade-off between Minimum and Average Margin
• Consistent Counterfactuals for Deep Models
• Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics
• Boosting the Certified Robustness of L-infinity Distance Nets
• Robust Models Are More Interpretable Because Attributions Look Normal
• How and When Adversarial Robustness Transfers in Knowledge Distillation?
• Enhancing Certified Robustness via Smoothed Weighted Ensembling
• Adversarial robustness via robust low rank representations
• Provable robustness against all adversarial $l_p$-perturbations for $p\ge 1$
• Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradients
• Relaxing Local Robustness
• Fast Geometric Projections for Local Robustness Certification
• Topological Effects on Attacks Against Vertex Classification
• A Distributionally Robust Optimization Method for Adversarial Multiple Kernel Learning
• Skew Orthogonal Convolutions
• On the Expected Complexity of Maxout Networks
• Recent Advances in Large Margin Learning
• Deterministic Certification to Adversarial Attacks via Bernstein Polynomial Approximation
• Training Provably Robust Models by Polyhedral Envelope Regularization