Statistically Robust Neural Network Classification
arXiv:1912.04884
Abstract
Despite their numerous successes, there are many scenarios where adversarial risk metrics do not provide an appropriate measure of robustness. For example, test-time perturbations may occur in a probabilistic manner rather than being generated by an explicit adversary, while the poor train--test generalization of adversarial metrics can limit their usage to simple problems. Motivated by this, we develop a probabilistic robust risk framework, the statistically robust risk (SRR), which considers pointwise corruption distributions, as opposed to worst-case adversaries. The SRR provides a distinct and complementary measure of robust performance, compared to natural and adversarial risk. We show that the SRR admits estimation and training schemes which are as simple and efficient as for the natural risk: these simply require noising the inputs, but with a principled derivation for exactly how and why this should be done. Furthermore, we demonstrate both theoretically and experimentally that it can provide superior generalization performance compared with adversarial risks, enabling application to high-dimensional datasets.
UAI 2021
References in corpus (11)
- Ensemble Adversarial Training: Attacks and Defenses
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Towards Deep Neural Network Architectures Robust to Adversarial Examples
- Certified Adversarial Robustness via Randomized Smoothing
- Adversarially Robust Generalization Requires More Data
- Towards the first adversarially robust neural network model on MNIST
- Certified Adversarial Robustness with Additive Noise
- Efficient Formal Safety Analysis of Neural Networks
- Adversarial Examples Are a Natural Consequence of Test Error in Noise
- Rademacher Complexity for Adversarially Robust Generalization
- Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distribution