Generalised Lipschitz Regularisation Equals Distributional Robustness
arXiv:2002.04197
Abstract
The problem of adversarial examples has highlighted the need for a theory of regularisation that is general enough to apply to exotic function classes, such as universal approximators. In response, we give a very general equality result regarding the relationship between distributional robustness and regularisation, as defined with a transportation cost uncertainty set. The theory allows us to (tightly) certify the robustness properties of a Lipschitz-regularised model with very mild assumptions. As a theoretical application we show a new result explicating the connection between adversarial learning and distributional robustness. We then give new results for how to achieve Lipschitz regularisation of kernel classifiers, which are demonstrated experimentally.
References in corpus (7)
- Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory
- Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
- Robust Wasserstein Profile Inference and Applications to Machine Learning
- Spectral Norm Regularization for Improving the Generalizability of Deep Learning
- Regularization via Mass Transportation
- A Kernel Perspective for Regularizing Deep Neural Networks
- An Approximate Shapley-Folkman Theorem