Lipschitz Networks and Distributional Robustness

Abstract

Robust risk minimisation has several advantages: it has been studied with regards to improving the generalisation properties of models and robustness to adversarial perturbation. We bound the distributionally robust risk for a model class rich enough to include deep neural networks by a regularised empirical risk involving the Lipschitz constant of the model. This allows us to interpretand quantify the robustness properties of a deep neural network. As an application we show the distributionally robust risk upperbounds the adversarial training risk.

References in corpus (12)

• Spectral Normalization for Generative Adversarial Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Certified Defenses against Adversarial Examples
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Robust Wasserstein Profile Inference and Applications to Machine Learning
• Spectral Norm Regularization for Improving the Generalizability of Deep Learning
• Spectrally-normalized margin bounds for neural networks
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• Regularization via Mass Transportation
• Revisiting Adversarial Risk
• Limitations of the Lipschitz constant as a defense against adversarial examples

Cited by in corpus (5)

• Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth
• Lipschitz regularized Deep Neural Networks generalize and are adversarially robust
• What training reveals about neural network complexity
• Nonparametric Online Learning Using Lipschitz Regularized Deep Neural Networks
• Principal Component Analysis Based on T${\ell }_1$-norm Maximization