Frank-Wolfe Methods in Probability Space
arXiv:2105.05352
Abstract
We introduce a new class of Frank-Wolfe algorithms for minimizing differentiable functionals over probability measures. This framework can be shown to encompass a diverse range of tasks in areas such as artificial intelligence, reinforcement learning, and optimization. Concrete computational complexities for these algorithms are established and demonstrate that these methods enjoy convergence in regimes that go beyond convexity and require minimal regularity of the underlying functional. Novel techniques used to obtain these results also lead to the development of new complexity bounds and duality theorems for a family of distributionally robust optimization problems. The performance of our method is demonstrated on several nonparametric estimation problems.
References in corpus (16)
- On Lazy Training in Differentiable Programming
- Conic Programming Reformulations of Two-Stage Distributionally Robust Linear Programs over Wasserstein Balls
- On the Convergence and Robustness of Training GANs with Regularized Optimal Transport
- Maximum Mean Discrepancy Gradient Flow
- Understanding and Accelerating Particle-Based Variational Inference
- Gradient descent algorithms for Bures-Wasserstein barycenters
- Nonconvex gradient flow in the Wasserstein metric and applications to constrained nonlocal interactions
- Optimal Transport Based Distributionally Robust Optimization: Structural Properties and Iterative Schemes
- Primal dual methods for Wasserstein gradient flows
- Smoothness and Stability in GANs
- A Distributionally Robust Approach to Fair Classification
- Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning
- Estimating Barycenters of Measures in High Dimensions
- Probability Functional Descent: A Unifying Perspective on GANs, Variational Inference, and Reinforcement Learning
- Efficient Stochastic Gradient Descent for Learning with Distributionally Robust Optimization
- Optimal Transport Relaxations with Application to Wasserstein GANs