From optimal transport to generative modeling: the VEGAN cookbook

Abstract

We study unsupervised generative modeling in terms of the optimal transport (OT) problem between true (but unknown) data distribution $P_X$ and the latent variable model distribution $P_G$. We show that the OT problem can be equivalently written in terms of probabilistic encoders, which are constrained to match the posterior and prior distributions over the latent space. When relaxed, this constrained optimization problem leads to a penalized optimal transport (POT) objective, which can be efficiently minimized using stochastic gradient descent by sampling from $P_X$ and $P_G$. We show that POT for the 2-Wasserstein distance coincides with the objective heuristically employed in adversarial auto-encoders (AAE) (Makhzani et al., 2016), which provides the first theoretical justification for AAEs known to the authors. We also compare POT to other popular techniques like variational auto-encoders (VAE) (Kingma and Welling, 2014). Our theoretical results include (a) a better understanding of the commonly observed blurriness of images generated by VAEs, and (b) establishing duality between Wasserstein GAN (Arjovsky and Bottou, 2017) and POT for the 1-Wasserstein distance.

References in corpus (3)

• Auto-Encoding Variational Bayes
• Adversarial Autoencoders
• Improved generator objectives for GANs

Cited by in corpus (45)

• Implicit Quantile Networks for Distributional Reinforcement Learning
• Wasserstein Auto-Encoders
• Diagnosing and Enhancing VAE Models
• Learning Generative Models with Sinkhorn Divergences
• Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model
• Learning to Simulate High Energy Particle Collisions from Unlabeled Data
• Convergence Problems with Generative Adversarial Networks (GANs)
• Improving GANs Using Optimal Transport
• COT-GAN: Generating Sequential Data via Causal Optimal Transport
• Deep Generative Models in the Real-World: An Open Challenge from Medical Imaging
• Eval all, trust a few, do wrong to none: Comparing sentence generation models
• Autoregressive Quantile Networks for Generative Modeling
• Regularized Optimal Transport and the Rot Mover's Distance
• Statistical and Topological Properties of Sliced Probability Divergences
• When OT meets MoM: Robust estimation of Wasserstein Distance
• Disentangled Recurrent Wasserstein Autoencoder
• Massively scalable Sinkhorn distances via the Nyström method
• Optimal Transport: Fast Probabilistic Approximation with Exact Solvers
• (Martingale) Optimal Transport And Anomaly Detection With Neural Networks: A Primal-dual Algorithm
• Approximating the Quadratic Transportation Metric in Near-Linear Time
• On Wasserstein Reinforcement Learning and the Fokker-Planck equation
• Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
• Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance
• Gaussian mixture models with Wasserstein distance
• Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions
• Importance Weighted Adversarial Variational Autoencoders for Spike Inference from Calcium Imaging Data
• Wasserstein-Wasserstein Auto-Encoders
• Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
• Adversarial Intrinsic Motivation for Reinforcement Learning
• A gradual, semi-discrete approach to generative network training via explicit Wasserstein minimization
• Deep Generative Learning via Variational Gradient Flow
• Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
• Investigating and Improving Latent Density Segmentation Models for Aleatoric Uncertainty Quantification in Medical Imaging
• Channel-Recurrent Autoencoding for Image Modeling
• Rate-Regularization and Generalization in VAEs
• Can VAEs Generate Novel Examples?
• Variance Constrained Autoencoding
• Learning and Inference in Imaginary Noise Models
• From Persistent Homology to Reinforcement Learning with Applications for Retail Banking
• Coupling Matrix Manifolds and Their Applications in Optimal Transport
• Informative GANs via Structured Regularization of Optimal Transport
• Visualization of AE's Training on Credit Card Transactions with Persistent Homology
• Bayesian Distributional Policy Gradients
• Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE
• Learning Deep-Latent Hierarchies by Stacking Wasserstein Autoencoders