Robustness Certificates Against Adversarial Examples for ReLU Networks
arXiv:1902.01235
Abstract
While neural networks have achieved high performance in different learning tasks, their accuracy drops significantly in the presence of small adversarial perturbations to inputs. Defenses based on regularization and adversarial training are often followed by new attacks to defeat them. In this paper, we propose attack-agnostic robustness certificates for a multi-label classification problem using a deep ReLU network. Although computing the exact distance of a given input sample to the classification decision boundary requires solving a non-convex optimization, we characterize two lower bounds for such distances, namely the simplex certificate and the decision boundary certificate. These robustness certificates leverage the piece-wise linear structure of ReLU networks and use the fact that in a polyhedron around a given sample, the prediction function is linear. In particular, the proposed simplex certificate has a closed-form, is differentiable and is an order of magnitude faster to compute than the existing methods even for deep networks. In addition to theoretical bounds, we provide numerical results for our certificates over MNIST and compare them with some existing upper bounds.
References in corpus (25)
- ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
- Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Synthesizing Robust Adversarial Examples
- Certified Defenses against Adversarial Examples
- The Space of Transferable Adversarial Examples
- Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
- Adversarial Machine Learning at Scale
- Efficient Neural Network Robustness Certification with General Activation Functions
- Certifying Some Distributional Robustness with Principled Adversarial Training
- Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
- Black-box Adversarial Attacks with Limited Queries and Information
- On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
- Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
- An approach to reachability analysis for feed-forward ReLU neural networks
- Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors
- Defensive Distillation is Not Robust to Adversarial Examples
- Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
- Towards Fast Computation of Certified Robustness for ReLU Networks
- MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
- Practical Black-Box Attacks against Machine Learning
- Universal adversarial perturbations
- EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
- Towards Robust Neural Networks via Random Self-ensemble
- Gradient Masking Causes CLEVER to Overestimate Adversarial Perturbation Size