Generalizable Adversarial Training via Spectral Normalization

Abstract

Deep neural networks (DNNs) have set benchmarks on a wide array of supervised learning tasks. Trained DNNs, however, often lack robustness to minor adversarial perturbations to the input, which undermines their true practicality. Recent works have increased the robustness of DNNs by fitting networks using adversarially-perturbed training samples, but the improved performance can still be far below the performance seen in non-adversarial settings. A significant portion of this gap can be attributed to the decrease in generalization performance due to adversarial training. In this work, we extend the notion of margin loss to adversarial settings and bound the generalization error for DNNs trained under several well-known gradient-based attack schemes, motivating an effective regularization scheme based on spectral normalization of the DNN's weight matrices. We also provide a computationally-efficient method for normalizing the spectral norm of convolutional layers with arbitrary stride and padding schemes in deep convolutional networks. We evaluate the power of spectral normalization extensively on combinations of datasets, network architectures, and adversarial training schemes. The code is available at https://github.com/jessemzhang/dl_spectral_normalization.

Cited by in corpus (17)

• Rademacher Complexity for Adversarially Robust Generalization
• Attacks Which Do Not Kill Training Make Adversarial Learning Stronger
• High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
• Globally-Robust Neural Networks
• Why Spectral Normalization Stabilizes GANs: Analysis and Improvements
• Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
• Learning Security Classifiers with Verified Global Robustness Properties
• Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective
• Train simultaneously, generalize better: Stability of gradient-based minimax learners
• Boosting the Certified Robustness of L-infinity Distance Nets
• On the expressivity of bi-Lipschitz normalizing flows
• A PAC-Bayes Analysis of Adversarial Robustness
• Adversarial Risk Bounds for Neural Networks through Sparsity based Compression
• Omni: Automated Ensemble with Unexpected Models against Adversarial Evasion Attack
• Towards Deeper Deep Reinforcement Learning with Spectral Normalization
• Robust Learning with Frequency Domain Regularization
• Generating Relevant Counter-Examples from a Positive Unlabeled Dataset for Image Classification