InfoCatVAE: Representation Learning with Categorical Variational Autoencoders
arXiv:1806.08240
Abstract
This paper describes InfoCatVAE, an extension of the variational autoencoder that enables unsupervised disentangled representation learning. InfoCatVAE uses multimodal distributions for the prior and the inference network and then maximizes the evidence lower bound objective (ELBO). We connect the new ELBO derived for our model with a natural soft clustering objective which explains the robustness of our approach. We then adapt the InfoGANs method to our setting in order to maximize the mutual information between the categorical code and the generated inputs and obtain an improved model.
arXiv comment9 pages, 3 appendix, 5 figures. arXiv admin note: text overlap with arXiv:1606.03657 by other authors
References in corpus (16)
- Categorical Reparameterization with Gumbel-Softmax
- Disentangling by Factorising
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
- InfoVAE: Information Maximizing Variational Autoencoders
- Hierarchical Multiscale Recurrent Neural Networks
- Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
- Isolating Sources of Disentanglement in Variational Autoencoders
- Structured Disentangled Representations
- On Unifying Deep Generative Models
- Recurrent Topic-Transition GAN for Visual Paragraph Generation
- Learning Disentangled Joint Continuous and Discrete Representations
- VAE with a VampPrior
- Nonparametric Variational Auto-encoders for Hierarchical Representation Learning
- A Survey of Inductive Biases for Factorial Representation-Learning
- Auto-Encoding Total Correlation Explanation