Lipschitz regularized Deep Neural Networks generalize and are adversarially robust
arXiv:1808.09540
Abstract
In this work we study input gradient regularization of deep neural networks, and demonstrate that such regularization leads to generalization proofs and improved adversarial robustness. The proof of generalization does not overcome the curse of dimensionality, but it is independent of the number of layers in the networks. The adversarial robustness regularization combines adversarial training, which we show to be equivalent to Total Variation regularization, with Lipschitz regularization. We demonstrate empirically that the regularized models are more robust, and that gradient norms of images can be used for attack detection.
arXiv comment18 pages, 4 figures (merged with arXiv:1810.00953)
References in corpus (12)
- Spectral Normalization for Generative Adversarial Networks
- Improved Regularization of Convolutional Neural Networks with Cutout
- Ensemble Adversarial Training: Attacks and Defenses
- Understanding deep learning requires rethinking generalization
- Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
- Robust Large Margin Deep Neural Networks
- Spectral Norm Regularization for Improving the Generalizability of Deep Learning
- Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
- Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability
- Sorting out Lipschitz function approximation
- Algorithms for Lipschitz Learning on Graphs
- Lipschitz Networks and Distributional Robustness
Cited by in corpus (9)
- A Closer Look at Accuracy vs. Robustness
- Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning
- Universal Lipschitz Approximation in Bounded Depth Neural Networks
- TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness
- The coupling effect of Lipschitz regularization in deep neural networks
- Lipschitz Bounds and Provably Robust Training by Laplacian Smoothing
- Enhancing Mixup-based Semi-Supervised Learning with Explicit Lipschitz Regularization
- Robusta: Robust AutoML for Feature Selection via Reinforcement Learning
- Depthwise Separable Convolutions Allow for Fast and Memory-Efficient Spectral Normalization