SOAR: Second-Order Adversarial Regularization

Abstract

Adversarial training is a common approach to improving the robustness of deep neural networks against adversarial examples. In this work, we propose a novel regularization approach as an alternative. To derive the regularizer, we formulate the adversarial robustness problem under the robust optimization framework and approximate the loss function using a second-order Taylor series expansion. Our proposed second-order adversarial regularizer (SOAR) is an upper bound based on the Taylor approximation of the inner-max in the robust optimization objective. We empirically show that the proposed method significantly improves the robustness of networks against the ${\ell }_{\infty }$ and ${\ell }_2$ bounded perturbations generated using cross-entropy-based PGD on CIFAR-10 and SVHN.

References in corpus (24)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Wide Residual Networks
• Theoretically Principled Trade-off between Robustness and Accuracy
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Adversarially Robust Generalization Requires More Data
• Fast is better than free: Revisiting adversarial training
• The Space of Transferable Adversarial Examples
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Learning with a Strong Adversary
• Adversarial Robustness through Local Linearization
• Simple Black-box Adversarial Attacks
• advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
• On Adaptive Attacks to Adversarial Example Defenses
• Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
• MMA Training: Direct Input Space Margin Maximization through Adversarial Training
• A Closer Look at Accuracy vs. Robustness
• Adversarial Manipulation of Deep Representations
• First-order Adversarial Vulnerability of Neural Networks and Input Dimension
• Wasserstein Adversarial Examples via Projected Sinkhorn Iterations
• Batch Normalization is a Cause of Adversarial Vulnerability

Cited by in corpus (2)

• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• Robust and Information-theoretically Safe Bias Classifier against Adversarial Attacks