A Primer on Multi-Neuron Relaxation-based Adversarial Robustness Certification

Abstract

The existence of adversarial examples poses a real danger when deep neural networks are deployed in the real world. The go-to strategy to quantify this vulnerability is to evaluate the model against specific attack algorithms. This approach is however inherently limited, as it says little about the robustness of the model against more powerful attacks not included in the evaluation. We develop a unified mathematical framework to describe relaxation-based robustness certification methods, which go beyond adversary-specific robustness evaluation and instead provide provable robustness guarantees against attacks by any adversary. We discuss the fundamental limitations posed by single-neuron relaxations and show how the recent ``k-ReLU'' multi-neuron relaxation framework of Singh et al. (2019) obtains tighter correlation-aware activation bounds by leveraging additional relational constraints among groups of neurons. Specifically, we show how additional pre-activation bounds can be mapped to corresponding post-activation bounds and how they can in turn be used to obtain tighter robustness certificates. We also present an intuitive way to visualize different relaxation-based certification methods. By approximating multiple non-linearities jointly instead of separately, the k-ReLU method is able to bypass the convex barrier imposed by single neuron relaxations.

References in corpus (14)

• Explaining and Harnessing Adversarial Examples
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Certified Defenses against Adversarial Examples
• Detecting Adversarial Samples from Artifacts
• On the (Statistical) Detection of Adversarial Examples
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
• An approach to reachability analysis for feed-forward ReLU neural networks
• A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
• Provably Minimally-Distorted Adversarial Examples
• The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification
• Adversarial Training is a Form of Data-dependent Operator Norm Regularization
• On Certifying Non-uniform Bound against Adversarial Attacks