Robustness of classifiers: from adversarial to random noise

Abstract

Several recent works have shown that state-of-the-art classifiers are vulnerable to worst-case (i.e., adversarial) perturbations of the datapoints. On the other hand, it has been empirically observed that these same classifiers are relatively robust to random noise. In this paper, we propose to study a \textit{semi-random} noise regime that generalizes both the random and worst-case noise regimes. We propose the first quantitative analysis of the robustness of nonlinear classifiers in this general noise regime. We establish precise theoretical bounds on the robustness of classifiers in this general regime, which depend on the curvature of the classifier's decision boundary. Our bounds confirm and quantify the empirical observations that classifiers satisfying curvature constraints are robust to random noise. Moreover, we quantify the robustness of classifiers in terms of the subspace dimension in the semi-random noise regime, and show that our bounds remarkably interpolate between the worst-case and random noise regimes. We perform experiments and show that the derived bounds provide very accurate estimates when applied to various state-of-the-art deep neural networks and datasets. This result suggests bounds on the curvature of the classifiers' decision boundaries that we support experimentally, and more generally offers important insights onto the geometry of high dimensional classification problems.

arXiv commentAccepted to NIPS 2016

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Robustness and Regularization of Support Vector Machines
• Learning with a Strong Adversary
• Foveation-based Mechanisms Alleviate Adversarial Examples
• Suppressing the Unusual: towards Robust CNNs using Symmetric Activation Functions

Cited by in corpus (33)

• Adversarial Examples Are Not Bugs, They Are Features
• Robustness May Be at Odds with Accuracy
• Deep Variational Information Bottleneck
• A General Framework for Adversarial Examples with Objectives
• Rademacher Complexity for Adversarially Robust Generalization
• A Review of Adversarial Attack and Defense for Classification Methods
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Analyzing the Robustness of Nearest Neighbors to Adversarial Examples
• A Survey On Universal Adversarial Attack
• Towards robust sensing for Autonomous Vehicles: An adversarial perspective
• Robust in Practice: Adversarial Attacks on Quantum Machine Learning
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks
• Arms Race in Adversarial Malware Detection: A Survey
• Divide, Denoise, and Defend against Adversarial Attacks
• Uncertainty Injection: A Deep Learning Method for Robust Optimization
• Evading classifiers in discrete domains with provable optimality guarantees
• Randomization matters. How to defend against strong adversarial attacks
• Towards Analyzing Semantic Robustness of Deep Neural Networks
• Achieving Adversarial Robustness via Sparsity
• How adversarial attacks can disrupt seemingly stable accurate classifiers
• Adversarial Robustness via Label-Smoothing
• GeoDA: a geometric framework for black-box adversarial attacks
• On the Effect of Low-Rank Weights on Adversarial Robustness of Neural Networks
• Certifying Incremental Quadratic Constraints for Neural Networks via Convex Optimization
• Lipschitz Bounds and Provably Robust Training by Laplacian Smoothing
• Certifying Neural Network Robustness to Random Input Noise from Samples
• Implicit Generative Modeling of Random Noise during Training for Adversarial Robustness
• Adversarial Robustness Guarantees for Random Deep Neural Networks
• Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective
• Noisy Feature Mixup
• Adversarial Detection by Approximation of Ensemble Boundary
• Comparative Study on Noise-Augmented Training and its Effect on Adversarial Robustness in ASR Systems