Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions

Abstract

By building upon the recent theory that established the connection between implicit generative modeling (IGM) and optimal transport, in this study, we propose a novel parameter-free algorithm for learning the underlying distributions of complicated datasets and sampling from them. The proposed algorithm is based on a functional optimization problem, which aims at finding a measure that is close to the data distribution as much as possible and also expressive enough for generative modeling purposes. We formulate the problem as a gradient flow in the space of probability measures. The connections between gradient flows and stochastic differential equations let us develop a computationally efficient algorithm for solving the optimization problem. We provide formal theoretical analysis where we prove finite-time error guarantees for the proposed algorithm. To the best of our knowledge, the proposed algorithm is the first nonparametric IGM algorithm with explicit theoretical guarantees. Our experimental results support our theory and show that our algorithm is able to successfully capture the structure of different types of data distributions.

arXiv commentPublished at the International Conference on Machine Learning (ICML) 2019

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Wasserstein Auto-Encoders
• Learning in Implicit Generative Models
• From optimal transport to generative modeling: the VEGAN cookbook
• Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model
• GAN and VAE from an Optimal Transport Point of View
• A Geometric View of Optimal Transportation and Generative Model
• Relaxed Wasserstein with Applications to GANs
• Asynchronous Stochastic Quasi-Newton MCMC for Non-Convex Optimization
• Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization

Cited by in corpus (5)

• Topological Learning for Brain Networks
• Learning a Domain-Invariant Embedding for Unsupervised Domain Adaptation Using Class-Conditioned Distribution Alignment
• Sliced Iterative Normalizing Flows
• Towards Optimal Randomized Strategies in Adversarial Example Game
• Dynamic Flows on Curved Space Generated by Labeled Data