Stochastic Variational Inference
arXiv:1206.7051
Abstract
We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Using stochastic variational inference, we analyze several large collections of documents: 300K articles from Nature, 1.8M articles from The New York Times, and 3.8M articles from Wikipedia. Stochastic inference can easily handle data sets of this size and outperforms traditional variational inference, which can only handle a smaller subset. (We also show that the Bayesian nonparametric topic model outperforms its parametric counterpart.) Stochastic variational inference lets us apply complex Bayesian models to massive data sets.
References in corpus (8)
- Inferring Parameters and Structure of Latent Variable Models by Variational Bayes
- On Smoothing and Inference for Topic Models
- Continuous Time Dynamic Topic Models
- Bayesian Nonparametric Inference of Switching Linear Dynamical Systems
- Nested Hierarchical Dirichlet Processes
- Variational Bayesian Inference with Stochastic Search
- Sparse Stochastic Inference for Latent Dirichlet allocation
- Nonparametric variational inference
Cited by in corpus (13)
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Automated Variational Inference in Probabilistic Programming
- Nested Variational Compression in Deep Gaussian Processes
- Differentiable Strong Lensing: Uniting Gravity and Neural Nets through Differentiable Probabilistic Programming
- Scalable Bayesian Inference for Excitatory Point Process Networks
- Variational inference for sparse spectrum Gaussian process regression
- Bayesian Incremental Learning for Deep Neural Networks
- A Nested HDP for Hierarchical Topic Models
- Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation
- Integrating Markov processes with structural causal modeling enables counterfactual inference in complex systems
- Community Detection in Weighted Multilayer Networks with Ambient Noise
- Particle Flow Bayes' Rule
- Training Restricted Boltzmann Machines with Binary Synapses using the Bayesian Learning Rule