On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
arXiv:1810.12715
Abstract
Recent work has shown that it is possible to train deep neural networks that are provably robust to norm-bounded adversarial perturbations. Most of these methods are based on minimizing an upper bound on the worst-case loss over all possible adversarial perturbations. While these techniques show promise, they often result in difficult optimization procedures that remain hard to scale to larger networks. Through a comprehensive analysis, we show how a simple bounding technique, interval bound propagation (IBP), can be exploited to train large provably robust neural networks that beat the state-of-the-art in verified accuracy. While the upper bound computed by IBP can be quite weak for general networks, we demonstrate that an appropriate loss and clever hyper-parameter schedule allow the network to adapt such that the IBP bound is tight. This results in a fast and stable learning algorithm that outperforms more sophisticated methods and achieves state-of-the-art results on MNIST, CIFAR-10 and SVHN. It also allows us to train the largest model to be verified beyond vacuous bounds on a downscaled version of ImageNet.
[v2] Best paper at NeurIPS SECML 2018 Workshop [v4] Accepted at ICCV 2019 under the title "Scalable Verified Training for Provably Robust Image Classification"
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Certified Defenses against Adversarial Examples
- Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
- Evaluating Robustness of Neural Networks with Mixed Integer Programming
- Formal Security Analysis of Neural Networks using Symbolic Intervals
- A Dual Approach to Scalable Verification of Deep Networks
- Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
- Provably Minimally-Distorted Adversarial Examples
- Training verified learners with learned verifiers
Cited by in corpus (127)
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
- On Evaluating Adversarial Robustness
- Fast is better than free: Revisiting adversarial training
- Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
- Are Labels Required for Improving Adversarial Robustness?
- advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
- Blind Backdoors in Deep Learning Models
- Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
- Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond
- Adversarial Training against Location-Optimized Adversarial Patches
- Certified Defenses for Adversarial Patches
- Adversarial Robustness Against the Union of Multiple Perturbation Models
- (De)Randomized Smoothing for Certifiable Defense against Patch Attacks
- Unlabeled Data Improves Adversarial Robustness
- MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
- Reachability Analysis of Neural Feedback Loops
- Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers
- Robust Reinforcement Learning on State Observations with Learned Optimal Adversary
- Overfitting in adversarially robust deep learning
- Optimal Provable Robustness of Quantum Classification via Quantum Hypothesis Testing
- Learning perturbation sets for robust machine learning
- Towards Stable and Efficient Training of Verifiably Robust Neural Networks
- Robust Multi-agent Communication via Multi-view Message Certification
- Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
- Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
- Denoised Smoothing: A Provable Defense for Pretrained Classifiers
- RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on states
- On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
- Robustness Verification for Transformers
- Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing
- HYDRA: Pruning Adversarially Robust Neural Networks
- Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble
- Randomized Smoothing of All Shapes and Sizes
- Towards Compact and Robust Deep Neural Networks
- Adversarial Machine Learning: Bayesian Perspectives
- Interval Universal Approximation for Neural Networks
- Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
- Verifying Individual Fairness in Machine Learning Models
- Improved Branch and Bound for Neural Network Verification via Lagrangian Decomposition
- Breaking certified defenses: Semantic adversarial examples with spoofed robustness certificates
- Certified Robustness to Adversarial Word Substitutions
- Wasserstein Smoothing: Certified Robustness against Wasserstein Adversarial Attacks
- A Provable Defense for Deep Residual Networks
- Random Smoothing Might be Unable to Certify Robustness for High-Dimensional Images
- Robust Deep Reinforcement Learning through Adversarial Loss
- Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming
- Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers
- Towards Understanding the Adversarial Vulnerability of Skeleton-based Action Recognition
- Deep Partition Aggregation: Provable Defense against General Poisoning Attacks
- Enhancing Gradient-based Attacks with Symbolic Intervals
- Adversarial Examples on Object Recognition: A Comprehensive Survey
- Manifold Regularization for Locally Stable Deep Neural Networks
- Certifying Confidence via Randomized Smoothing
- On Certifying Non-uniform Bound against Adversarial Attacks
- Cost-Aware Robust Tree Ensembles for Security Applications
- Improved, Deterministic Smoothing for L_1 Certified Robustness
- Robust Encodings: A Framework for Combating Adversarial Typos
- Understanding Adversarial Robustness: The Trade-off between Minimum and Average Margin
- Bayesian Inference with Certifiable Adversarial Robustness
- Jacobian Adversarially Regularized Networks for Robustness
- Bit Error Robustness for Energy-Efficient DNN Accelerators
- Imitation Learning with Stability and Safety Guarantees
- Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
- On Training Robust PDF Malware Classifiers
- Improving Certified Robustness via Statistical Learning with Logical Reasoning
- Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers
- Boosting the Certified Robustness of L-infinity Distance Nets
- Fast Certified Robust Training with Short Warmup
- Enhancing Certified Robustness via Smoothed Weighted Ensembling
- Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks
- Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselines
- Robustness Certificates Against Adversarial Examples for ReLU Networks
- Adversarial robustness via robust low rank representations
- Scaling the Convex Barrier with Sparse Dual Algorithms
- Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations
- Abstract Interpretation of Fixpoint Iterators with Applications to Neural Networks
- Detecting Localized Adversarial Examples: A Generic Approach using Critical Region Analysis
- Safe Predictors for Enforcing Input-Output Specifications
- Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
- Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey
- Detection as Regression: Certified Object Detection by Median Smoothing
- Sequential Randomized Smoothing for Adversarially Robust Speech Recognition
- Provably Robust Detection of Out-of-distribution Data (almost) for free
- CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
- Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds
- An efficient nonconvex reformulation of stagewise convex optimization problems
- Policy Smoothing for Provably Robust Reinforcement Learning
- Differentially Private Adversarial Robustness Through Randomized Perturbations
- Auditing AI models for Verified Deployment under Semantic Specifications
- Automatic Fairness Testing of Neural Classifiers through Adversarial Sampling
- Neural Network Branch-and-Bound for Neural Network Verification
- Generating Adversarial Examples with Graph Neural Networks
- GoTube: Scalable Stochastic Verification of Continuous-Depth Models
- Understanding Catastrophic Overfitting in Adversarial Training
- Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons
- Bridging the Gap Between Adversarial Robustness and Optimization Bias
- Tight Second-Order Certificates for Randomized Smoothing
- On the Tightness of Semidefinite Relaxations for Certifying Robustness to Adversarial Examples
- Provable robustness against all adversarial -perturbations for
- Universal Approximation with Certified Networks
- What it Thinks is Important is Important: Robustness Transfers through Input Gradients
- HyperCube: Implicit Field Representations of Voxelized 3D Models
- Fast Geometric Projections for Local Robustness Certification
- Connecting Lyapunov Control Theory to Adversarial Attacks
- Provably-Robust Runtime Monitoring of Neuron Activation Patterns
- Robust Stability Analysis of Positive Lure System with Neural Network Feedback
- Fast and Stable Interval Bounds Propagation for Training Verifiably Robust Models
- Perceptually Constrained Adversarial Attacks
- Probabilistic Safety for Bayesian Neural Networks
- Domain Invariant Adversarial Learning
- An automatic differentiation system for the age of differential privacy
- A Multiclass Boosting Framework for Achieving Fast and Provable Adversarial Robustness
- Provably robust deep generative models
- Certifying Strategyproof Auction Networks
- Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning
- Learning to Separate Clusters of Adversarial Representations for Robust Adversarial Detection
- Improving the Certified Robustness of Neural Networks via Consistency Regularization
- Training Efficiency and Robustness in Deep Learning
- Towards Assessment of Randomized Smoothing Mechanisms for Certifying Adversarial Robustness
- Robust Vision-Based Cheat Detection in Competitive Gaming
- Simpler Certified Radius Maximization by Propagating Covariances
- Expected Tight Bounds for Robust Training
- Reachability Analysis of Neural Networks with Uncertain Parameters
- Achieving Model Robustness through Discrete Adversarial Training
- Training Provably Robust Models by Polyhedral Envelope Regularization
- Deterministic Certification to Adversarial Attacks via Bernstein Polynomial Approximation
- Adversarial Robustness Across Representation Spaces