Consistency Regularization for Certified Robustness of Smoothed Classifiers

Abstract

A recent technique of randomized smoothing has shown that the worst-case (adversarial) ${\ell }_2$-robustness can be transformed into the average-case Gaussian-robustness by "smoothing" a classifier, i.e., by considering the averaged prediction over Gaussian noise. In this paradigm, one should rethink the notion of adversarial robustness in terms of generalization ability of a classifier under noisy observations. We found that the trade-off between accuracy and certified robustness of smoothed classifiers can be greatly controlled by simply regularizing the prediction consistency over noise. This relationship allows us to design a robust training objective without approximating a non-existing smoothed classifier, e.g., via soft smoothing. Our experiments under various deep neural network architectures and datasets show that the "certified" ${\ell }_2$-robustness can be dramatically improved with the proposed regularization, even achieving better or comparable results to the state-of-the-art approaches with significantly less training costs and hyperparameters.

arXiv comment19 pages; NeurIPS 2020; Code is available at https://github.com/jh-jeong/smoothing-consistency

References in corpus (16)

• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Certified Adversarial Robustness via Randomized Smoothing
• On Evaluating Adversarial Robustness
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Certified Adversarial Robustness with Additive Noise
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Adversarial Training and Robustness for Multiple Perturbations
• Testing Robustness Against Unforeseen Adversaries
• MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
• A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions

Cited by in corpus (3)

• TSS: Transformation-Specific Smoothing for Robustness Certification
• CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
• Multi-head Ensemble of Smoothed Classifiers for Certified Robustness