Second-Order Provable Defenses against Adversarial Attacks

Abstract

A robustness certificate is the minimum distance of a given input to the decision boundary of the classifier (or its lower bound). For {\it any} input perturbations with a magnitude smaller than the certificate value, the classification output will provably remain unchanged. Exactly computing the robustness certificates for neural networks is difficult since it requires solving a non-convex optimization. In this paper, we provide computationally-efficient robustness certificates for neural networks with differentiable activation functions in two steps. First, we show that if the eigenvalues of the Hessian of the network are bounded, we can compute a robustness certificate in the $l_2$ norm efficiently using convex optimization. Second, we derive a computationally-efficient differentiable upper bound on the curvature of a deep network. We also use the curvature bound as a regularization term during the training of the network to boost its certified robustness. Putting these results together leads to our proposed {\bf C}urvature-based {\bf R}obustness {\bf C}ertificate (CRC) and {\bf C}urvature-based {\bf R}obust {\bf T}raining (CRT). Our numerical results show that CRT leads to significantly higher certified robust accuracy compared to interval-bound propagation (IBP) based training. We achieve certified robust accuracy 69.79\%, 57.78\% and 53.19\% while IBP-based methods achieve 44.96\%, 44.74\% and 44.66\% on 2,3 and 4 layer networks respectively on the MNIST-dataset.

References in corpus (13)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Adversarial Machine Learning at Scale
• Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
• An approach to reachability analysis for feed-forward ReLU neural networks
• Semidefinite relaxations for certifying robustness to adversarial examples
• On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
• Provably Minimally-Distorted Adversarial Examples
• Improving the Robustness of Deep Neural Networks via Stability Training
• Sorting out Lipschitz function approximation
• Towards Robust Neural Networks via Random Self-ensemble

Cited by in corpus (2)

• Improved, Deterministic Smoothing for L_1 Certified Robustness
• Group-Structured Adversarial Training