From optimal transport to generative modeling: the VEGAN cookbook

Abstract

We study unsupervised generative modeling in terms of the optimal transport (OT) problem between true (but unknown) data distribution $P_X$ and the latent variable model distribution $P_G$. We show that the OT problem can be equivalently written in terms of probabilistic encoders, which are constrained to match the posterior and prior distributions over the latent space. When relaxed, this constrained optimization problem leads to a penalized optimal transport (POT) objective, which can be efficiently minimized using stochastic gradient descent by sampling from $P_X$ and $P_G$. We show that POT for the 2-Wasserstein distance coincides with the objective heuristically employed in adversarial auto-encoders (AAE) (Makhzani et al., 2016), which provides the first theoretical justification for AAEs known to the authors. We also compare POT to other popular techniques like variational auto-encoders (VAE) (Kingma and Welling, 2014). Our theoretical results include (a) a better understanding of the commonly observed blurriness of images generated by VAEs, and (b) establishing duality between Wasserstein GAN (Arjovsky and Bottou, 2017) and POT for the 1-Wasserstein distance.

References in corpus (1)

• Improved generator objectives for GANs

Cited by in corpus (16)

• Learning Generative Models with Sinkhorn Divergences
• Disentangled Recurrent Wasserstein Autoencoder
• (Martingale) Optimal Transport And Anomaly Detection With Neural Networks: A Primal-dual Algorithm
• On Wasserstein Reinforcement Learning and the Fokker-Planck equation
• Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
• Wasserstein-Wasserstein Auto-Encoders
• Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
• A gradual, semi-discrete approach to generative network training via explicit Wasserstein minimization
• Deep Generative Learning via Variational Gradient Flow
• Can VAEs Generate Novel Examples?
• Learning and Inference in Imaginary Noise Models
• Variance Constrained Autoencoding
• Informative GANs via Structured Regularization of Optimal Transport
• Coupling Matrix Manifolds and Their Applications in Optimal Transport
• Bayesian Distributional Policy Gradients
• From Persistent Homology to Reinforcement Learning with Applications for Retail Banking