Adversarially Robust Kernel Smoothing
arXiv:2102.08474
Abstract
We propose a scalable robust learning algorithm combining kernel smoothing and robust optimization. Our method is motivated by the convex analysis perspective of distributionally robust optimization based on probability metrics, such as the Wasserstein distance and the maximum mean discrepancy. We adapt the integral operator using supremal convolution in convex analysis to form a novel function majorant used for enforcing robustness. Our method is simple in form and applies to general loss functions and machine learning models. Exploiting a connection with optimal transport, we prove theoretical guarantees for certified robustness under distribution shift. Furthermore, we report experiments with general machine learning models, such as deep neural networks, to demonstrate competitive performance with the state-of-the-art certifiable robust learning algorithms based on the Wasserstein distance.
References in corpus (18)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Deep Residual Learning for Image Recognition
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Certifying Some Distributional Robustness with Principled Adversarial Training
- Robustness and Regularization of Support Vector Machines
- Robust Wasserstein Profile Inference and Applications to Machine Learning
- The generalization error of random features regression: Precise asymptotics and double descent curve
- Distributionally Robust Logistic Regression
- Regularization via Mass Transportation
- On the Convergence Proof of AMSGrad and a New Version
- Metric Learning for Adversarial Robustness
- On gradient regularizers for MMD GANs
- A Kernel Perspective for Regularizing Deep Neural Networks
- Optimal Transport Based Distributionally Robust Optimization: Structural Properties and Iterative Schemes
- Kernel regression in high dimensions: Refined analysis beyond double descent
- Kernel Distributionally Robust Optimization
- Distributional Robustness with IPMs and links to Regularization and GANs