A New Distribution on the Simplex with Auto-Encoding Applications
arXiv:1905.12052
Abstract
We construct a new distribution for the simplex using the Kumaraswamy distribution and an ordered stick-breaking process. We explore and develop the theoretical properties of this new distribution and prove that it exhibits symmetry under the same conditions as the well-known Dirichlet. Like the Dirichlet, the new distribution is adept at capturing sparsity but, unlike the Dirichlet, has an exact and closed form reparameterization--making it well suited for deep variational Bayesian modeling. We demonstrate the distribution's utility in a variety of semi-supervised auto-encoding tasks. In all cases, the resulting models achieve competitive performance commensurate with their simplicity, use of explicit probability models, and abstinence from adversarial training.
15 pages, 6 figures, 1 tables
References in corpus (10)
- Auto-Encoding Variational Bayes
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Semi-Supervised Learning with Deep Generative Models
- MixMatch: A Holistic Approach to Semi-Supervised Learning
- Adversarial Autoencoders
- Autoencoding Variational Inference For Topic Models
- Implicit Reparameterization Gradients
- Pathwise Derivatives Beyond the Reparameterization Trick
- PixelGAN Autoencoders
- Inferencing Based on Unsupervised Learning of Disentangled Representations