Fast Variational Inference in the Conjugate Exponential Family

Abstract

We present a general method for deriving collapsed variational inference algo- rithms for probabilistic models in the conjugate exponential family. Our method unifies many existing approaches to collapsed variational inference. Our collapsed variational inference leads to a new lower bound on the marginal likelihood. We exploit the information geometry of the bound to derive much faster optimization methods based on conjugate gradients for these models. Our approach is very general and is easily applied to any model where the mean field update equations have been derived. Empirically we show significant speed-ups for probabilistic models optimized using our bound.

Accepted at NIPS 2012

References in corpus (2)

• On Smoothing and Inference for Topic Models
• Identifying differentially expressed transcripts from RNA-seq data with biological variation

Cited by in corpus (27)

• Variational Inference: A Review for Statisticians
• Gaussian Processes for Big Data
• Approximate Bayesian inference as a gauge theory
• Breaking Sticks and Ambiguities with Adaptive Skip-gram
• When Gaussian Process Meets Big Data: A Review of Scalable GPs
• Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
• Scalable Generalized Dynamic Topic Models
• Advances in Variational Inference
• Probabilistic Line Searches for Stochastic Optimization
• Variational Bayesian Approximation of Inverse Problems using Sparse Precision Matrices
• Fast Second-Order Stochastic Backpropagation for Variational Inference
• GP-select: Accelerating EM using adaptive subspace preselection
• Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization
• Improving predictions of Bayesian neural nets via local linearization
• BayesPy: Variational Bayesian Inference in Python
• Continual Multi-task Gaussian Processes
• BasisVAE: Translation-invariant feature-level clustering with Variational Autoencoders
• Spatio-Temporal Variational Gaussian Processes
• Fast Approximate Inference of Transcript Expression Levels from RNA-seq Data
• Collapsed Variational Bayes Inference of Infinite Relational Model
• Myopic Bayesian Design of Experiments via Posterior Sampling and Probabilistic Programming
• Extreme Stochastic Variational Inference: Distributed and Asynchronous
• Disentangling the Gauss-Newton Method and Approximate Inference for Neural Networks
• A Probabilistic Representation of Deep Learning
• Heteroscedastic Relevance Vector Machine
• Regularized Sparse Gaussian Processes
• Fast and accurate approximate inference of transcript expression from RNA-seq data