The Variational Gaussian Process
arXiv:1511.06499
Abstract
Variational inference is a powerful tool for approximate inference, and it has been recently applied for representation learning with deep generative models. We develop the variational Gaussian process (VGP), a Bayesian nonparametric variational family, which adapts its shape to match complex posterior distributions. The VGP generates approximate posterior samples by generating latent inputs and warping them through random non-linear mappings; the distribution over random mappings is learned during inference, enabling the transformed outputs to adapt to varying complexity. We prove a universal approximation theorem for the VGP, demonstrating its representative power for learning any model. For inference we present a variational objective inspired by auto-encoders and perform black box inference over a wide class of models. The VGP achieves new state-of-the-art results for unsupervised learning, inferring models such as the deep latent Gaussian model and the recently proposed DRAW.
Appears in International Conference on Learning Representations, 2016
References in corpus (4)
Cited by in corpus (10)
- Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
- Ladder Variational Autoencoders
- Automatic Differentiation Variational Inference
- BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
- Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
- Overdispersed Black-Box Variational Inference
- Semi-Supervised Generation with Cluster-aware Generative Models
- On the Necessity and Effectiveness of Learning the Prior of Variational Auto-Encoder
- Constructing the Matrix Multilayer Perceptron and its Application to the VAE
- Variational Marginal Particle Filters