Learning Deep Generative Models with Doubly Stochastic MCMC
arXiv:1506.04557
Abstract
We present doubly stochastic gradient MCMC, a simple and generic method for (approximate) Bayesian inference of deep generative models (DGMs) in a collapsed continuous parameter space. At each MCMC sampling step, the algorithm randomly draws a mini-batch of data samples to estimate the gradient of log-posterior and further estimates the intractable expectation over hidden variables via a neural adaptive importance sampler, where the proposal distribution is parameterized by a deep neural network and learnt jointly. We demonstrate the effectiveness on learning various DGMs in a wide range of tasks, including density estimation, data generation and missing data imputation. Our method outperforms many state-of-the-art competitors.
References in corpus (12)
- Auto-Encoding Variational Bayes
- Theano: new features and speed improvements
- Stochastic Gradient Hamiltonian Monte Carlo
- Deep Generative Stochastic Networks Trainable by Backprop
- Importance Weighted Autoencoders
- Neural Variational Inference and Learning in Belief Networks
- Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring
- Deep AutoRegressive Networks
- A Deep and Tractable Density Estimator
- Generalized Beta Mixtures of Gaussians
- Learning the Structure of Deep Sparse Graphical Models
- Iterative Neural Autoregressive Distribution Estimator (NADE-k)