Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers

Abstract

Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to ${\ell }_2$-norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adapted attack for smoothed classifiers, and we show how this attack can be used in an adversarial training setting to boost the provable robustness of smoothed classifiers. We demonstrate through extensive experimentation that our method consistently outperforms all existing provably ${\ell }_2$-robust classifiers by a significant margin on ImageNet and CIFAR-10, establishing the state-of-the-art for provable ${\ell }_2$-defenses. Moreover, we find that pre-training and semi-supervised learning boost adversarially trained smoothed classifiers even further. Our code and trained models are available at http://github.com/Hadisalman/smoothing-adversarial .

Spotlight at the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada; 9 pages main text; 31 pages total

References in corpus (17)

• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• Certified Defenses against Adversarial Examples
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Evaluating Robustness of Neural Networks with Mixed Integer Programming
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• An approach to reachability analysis for feed-forward ReLU neural networks
• Semidefinite relaxations for certifying robustness to adversarial examples
• Efficient Formal Safety Analysis of Neural Networks
• A Dual Approach to Scalable Verification of Deep Networks
• A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
• On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
• Training verified learners with learned verifiers
• Provable Robustness of ReLU networks via Maximization of Linear Regions
• Deep Neural Networks as 0-1 Mixed Integer Linear Programs: A Feasibility Study

Cited by in corpus (90)

• Fast is better than free: Revisiting adversarial training
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• Do Adversarially Robust ImageNet Models Transfer Better?
• Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond
• Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
• Adversarial Self-Supervised Contrastive Learning
• A Closer Look at Accuracy vs. Robustness
• Towards quantum enhanced adversarial robustness in machine learning
• PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximations
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• (De)Randomized Smoothing for Certifiable Defense against Patch Attacks
• Boosting Adversarial Training with Hypersphere Embedding
• MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
• On Certifying Robustness against Backdoor Attacks via Randomized Smoothing
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack
• Towards Stable and Efficient Training of Verifiably Robust Neural Networks
• PatchGuard: A Provably Robust Defense against Adversarial Patches via Small Receptive Fields and Masking
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness
• Denoised Smoothing: A Provable Defense for Pretrained Classifiers
• On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Adversarial Distributional Training for Robust Deep Learning
• HYDRA: Pruning Adversarially Robust Neural Networks
• Randomized Smoothing of All Shapes and Sizes
• Globally-Robust Neural Networks
• Learning Security Classifiers with Verified Global Robustness Properties
• Robust Neural Networks using Randomized Adversarial Training
• Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Framework
• Constrained Learning with Non-Convex Losses
• Consistency Regularization for Certified Robustness of Smoothed Classifiers
• PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image Classifier
• Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation
• Breaking certified defenses: Semantic adversarial examples with spoofed robustness certificates
• Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning
• Certifiably Robust Interpretation in Deep Learning
• Poisoned classifiers are not only backdoored, they are fundamentally broken
• Deep Partition Aggregation: Provable Defense against General Poisoning Attacks
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Calibrated Surrogate Losses for Adversarially Robust Classification
• On the Generalization Properties of Adversarial Training
• Improved, Deterministic Smoothing for L_1 Certified Robustness
• Certifying Confidence via Randomized Smoothing
• Efficient Robustness Certificates for Discrete Data: Sparsity-Aware Randomized Smoothing for Graphs, Images and More
• Certifiably-Robust Federated Adversarial Learning via Randomized Smoothing
• Random Noise Defense Against Query-Based Black-Box Attacks
• On the Certified Robustness for Ensemble Models and Beyond
• Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?
• Boosting the Certified Robustness of L-infinity Distance Nets
• TSS: Transformation-Specific Smoothing for Robustness Certification
• PointGuard: Provably Robust 3D Point Cloud Classification
• Improving Certified Robustness via Statistical Learning with Logical Reasoning
• Fast Certified Robust Training with Short Warmup
• Enhancing Certified Robustness via Smoothed Weighted Ensembling
• Adversarial robustness via robust low rank representations
• Higher-Order Certification for Randomized Smoothing
• Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations
• Data-Dependent Randomized Smoothing
• Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time
• Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselines
• Towards Robust Waveform-Based Acoustic Models
• NLP Verification: Towards a General Methodology for Certifying Robustness
• Detection as Regression: Certified Object Detection by Median Smoothing
• Certified Robustness of Graph Neural Networks against Adversarial Structural Perturbation
• CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• DetectorGuard: Provably Securing Object Detectors against Localized Patch Hiding Attacks
• On the Tightness of Semidefinite Relaxations for Certifying Robustness to Adversarial Examples
• Policy Smoothing for Provably Robust Reinforcement Learning
• Tight Second-Order Certificates for Randomized Smoothing
• A PAC-Bayes Analysis of Adversarial Robustness
• Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons
• Smoothed Inference for Adversarially-Trained Models
• Towards Deep Learning Models Resistant to Large Perturbations
• Certifiably Robust Variational Autoencoders
• Insta-RS: Instance-wise Randomized Smoothing for Improved Robustness and Accuracy
• ANCER: Anisotropic Certification via Sample-wise Volume Maximization
• Multi-head Ensemble of Smoothed Classifiers for Certified Robustness
• Provable Robust Classification via Learned Smoothed Densities
• Safety-Aware Hardening of 3D Object Detection Neural Network Systems
• Query complexity of adversarial attacks
• When are Non-Parametric Methods Robust?
• Dynamic Defense Approach for Adversarial Robustness in Deep Neural Networks via Stochastic Ensemble Smoothed Model
• Analysis of Random Perturbations for Robust Convolutional Neural Networks
• Bridged Adversarial Training
• Hidden Cost of Randomized Smoothing
• HRFA: High-Resolution Feature-based Attack
• Training Provably Robust Models by Polyhedral Envelope Regularization
• Intriguing Properties of Input-dependent Randomized Smoothing
• Boosting Certified Robustness for Time Series Classification with Efficient Self-Ensemble