A gradual, semi-discrete approach to generative network training via explicit Wasserstein minimization
arXiv:1906.03471
Abstract
This paper provides a simple procedure to fit generative networks to target distributions, with the goal of a small Wasserstein distance (or other optimal transport costs). The approach is based on two principles: (a) if the source randomness of the network is a continuous distribution (the "semi-discrete" setting), then the Wasserstein distance is realized by a deterministic optimal transport mapping; (b) given an optimal transport mapping between a generator network and a target distribution, the Wasserstein distance may be decreased via a regression between the generated data and the mapped target points. The procedure here therefore alternates these two steps, forming an optimal transport and regressing against it, gradually adjusting the generator network towards the target distribution. Mathematically, this approach is shown to minimize the Wasserstein distance to both the empirical target distribution, and also its underlying population counterpart. Empirically, good performance is demonstrated on the training and testing sets of the MNIST and Thin-8 data. The paper closes with a discussion of the unsuitability of the Wasserstein distance for certain tasks, as has been identified in prior work [Arora et al., 2017, Huang et al., 2017].
arXiv commentAppears in ICML 2019
References in corpus (10)
- Auto-Encoding Variational Bayes
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Improved Techniques for Training GANs
- The Cramer Distance as a Solution to Biased Wasserstein Gradients
- Wasserstein Auto-Encoders
- Stochastic Optimization for Large-scale Optimal Transport
- From optimal transport to generative modeling: the VEGAN cookbook
- GAN and VAE from an Optimal Transport Point of View
- Error bounds for approximations with deep ReLU networks