Gaussian mixture models with Wasserstein distance
arXiv:1806.04465
Abstract
Generative models with both discrete and continuous latent variables are highly motivated by the structure of many real-world data sets. They present, however, subtleties in training often manifesting in the discrete latent being under leveraged. In this paper, we show that such models are more amenable to training when using the Optimal Transport framework of Wasserstein Autoencoders. We find our discrete latent variable to be fully leveraged by the model when trained, without any modifications to the objective function or significant fine tuning. Our model generates comparable samples to other approaches while using relatively simple neural networks, since the discrete latent variable carries much of the descriptive burden. Furthermore, the discrete latent provides significant control over generation.
8 pages, 5 figures
References in corpus (14)
- Auto-Encoding Variational Bayes
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Categorical Reparameterization with Gumbel-Softmax
- MCMC using Hamiltonian dynamics
- Neural Discrete Representation Learning
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- Generative Moment Matching Networks
- Adversarial Autoencoders
- Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
- Composing graphical models with neural networks for structured representations and fast inference
- From optimal transport to generative modeling: the VEGAN cookbook