Adversarial Risk Bounds via Function Transformation

Abstract

We derive bounds for a notion of adversarial risk, designed to characterize the robustness of linear and neural network classifiers to adversarial perturbations. Specifically, we introduce a new class of function transformations with the property that the risk of the transformed functions upper-bounds the adversarial risk of the original functions. This reduces the problem of deriving bounds on the adversarial risk to the problem of deriving risk bounds using standard learning-theoretic techniques. We then derive bounds on the Rademacher complexities of the transformed function classes, obtaining error rates on the same order as the generalization error of the original function classes. We also discuss extensions of our theory to multiclass classification and regression. Finally, we provide two algorithms for optimizing the adversarial risk bounds in the linear case, and discuss connections to regularization and distributional robustness.

arXiv comment43 pages, 3 figures

References in corpus (7)

• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Robustness and Regularization of Support Vector Machines
• Adversarial examples from computational constraints
• Rademacher Complexity for Adversarially Robust Generalization
• Revisiting Adversarial Risk

Cited by in corpus (22)

• Adversarial Weight Perturbation Helps Robust Generalization
• Are Labels Required for Improving Adversarial Robustness?
• Adversarial Training Can Hurt Generalization
• VC Classes are Adversarially Robustly Learnable, but Only Improperly
• The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization
• Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
• Recent advances in deep learning theory
• More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Models
• Calibrated Surrogate Losses for Adversarially Robust Classification
• Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
• Calibration and Consistency of Adversarial Surrogate Losses
• Train simultaneously, generalize better: Stability of gradient-based minimax learners
• Sharp Statistical Guarantees for Adversarially Robust Gaussian Classification
• Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks
• Efficiently Learning Adversarially Robust Halfspaces with Noise
• Adversarial Risk Bounds for Neural Networks through Sparsity based Compression
• Robustness, Privacy, and Generalization of Adversarial Training
• Robust and Private Learning of Halfspaces
• Towards Deep Learning Models Resistant to Large Perturbations
• Domain Invariant Adversarial Learning
• Estimating Principal Components under Adversarial Perturbations
• Unique properties of adversarially trained linear classifiers on Gaussian data