Amortized Population Gibbs Samplers with Neural Sufficient Statistics
arXiv:1911.01382
Abstract
We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frames structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditional proposal by minimizing the inclusive KL divergence with respect to the conditional posterior. To appropriately account for the size of the input data, we develop a new parameterization in terms of neural sufficient statistics. Experiments show that APG samplers can train highly structured deep generative models in an unsupervised manner, and achieve substantial improvements in inference accuracy relative to standard autoencoding variational methods.
References in corpus (17)
- Auto-Encoding Variational Bayes
- MCMC using Hamiltonian dynamics
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Unsupervised Learning of Video Representations using LSTMs
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
- Auxiliary Deep Generative Models
- REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
- Variational inference for Monte Carlo objectives
- Monte Carlo Gradient Estimation in Machine Learning
- Nested Sequential Monte Carlo Methods
- Reweighted Wake-Sleep
- Approximate Inference with Amortised MCMC
- Meta-Learning MCMC Proposals
- Revisiting Reweighted Wake-Sleep for Models with Stochastic Control Flow
- Improving Explorability in Variational Inference with Annealed Variational Objectives
- Structured Neural Topic Models for Reviews