Structured Stochastic Variational Inference

Abstract

Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization. The algorithm relies on the use of fully factorized variational distributions. However, this "mean-field" independence approximation limits the fidelity of the posterior approximation, and introduces local optima. We show how to relax the mean-field approximation to allow arbitrary dependencies between global parameters and local hidden variables, producing better parameter estimates by reducing bias, sensitivity to local optima, and sensitivity to hyperparameters.

References in corpus (4)

• On Smoothing and Inference for Topic Models
• Streaming Variational Bayes
• Variational Bayesian Inference with Stochastic Search
• Sparse Stochastic Inference for Latent Dirichlet allocation

Cited by in corpus (25)

• Deep Probabilistic Programming
• Seismic tomography using variational inference methods
• Coordinated Multi-Agent Imitation Learning
• Copula variational inference
• Advances in Variational Inference
• Adversarial Message Passing For Graphical Models
• Hierarchical Variational Models
• Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes
• A trust-region method for stochastic variational inference with applications to streaming data
• Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning
• Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning
• Automatic variational inference with cascading flows
• GRADE: Graph Dynamic Embedding
• Perturb-and-MPM: Quantifying Segmentation Uncertainty in Dense Multi-Label CRFs
• An Empirical Study of Stochastic Variational Algorithms for the Beta Bernoulli Process
• Flexible mean field variational inference using mixtures of non-overlapping exponential families
• Doubly Semi-Implicit Variational Inference
• The Population Posterior and Bayesian Inference on Streams
• Deep Generative Model with Beta Bernoulli Process for Modeling and Learning Confounding Factors
• Random Function Priors for Correlation Modeling
• ADMM-based Networked Stochastic Variational Inference
• Hierarchical learning of grids of microtopics
• Relay Variational Inference: A Method for Accelerated Encoderless VI
• ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models
• Inter-Battery Topic Representation Learning