Streaming Variational Bayes

Abstract

We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to the estimated posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes (VB) as the primitive, by fitting the latent Dirichlet allocation model to two large-scale document collections. We demonstrate the advantages of our algorithm over stochastic variational inference (SVI) by comparing the two after a single pass through a known amount of data---a case where SVI may be applied---and in the streaming setting, where SVI does not apply.

25 pages, 3 figures, 1 table

References in corpus (4)

• Expectation Propagation for approximate Bayesian inference
• HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
• On Smoothing and Inference for Topic Models
• The Big Data Bootstrap

Cited by in corpus (39)

• Variational Federated Multi-Task Learning
• Lifelong Generative Modeling
• Hierarchical Optimal Transport for Document Representation
• Structured Stochastic Variational Inference
• Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL
• Stochastic Variational Inference for Hidden Markov Models
• Online Coreset Selection for Rehearsal-based Continual Learning
• Online Bayesian Passive-Aggressive Learning
• Decentralized Bayesian Learning over Graphs
• Advances in Variational Inference
• Probabilistic Line Searches for Stochastic Optimization
• Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
• Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
• Variational consensus Monte Carlo
• Probabilistic Graphical Models on Multi-Core CPUs using Java 8
• A trust-region method for stochastic variational inference with applications to streaming data
• A Generalization Bound for Online Variational Inference
• Streaming Gibbs Sampling for LDA Model
• Embarrassingly Parallel Variational Inference in Nonconjugate Models
• Approximate Decentralized Bayesian Inference
• Model-Parallel Inference for Big Topic Models
• Self-Supervised Deep Visual Odometry with Online Adaptation
• Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction
• Sparse Variational Inference: Bayesian Coresets from Scratch
• Streaming Adaptive Nonparametric Variational Autoencoder
• Variational online learning of neural dynamics
• Low Complexity Approximate Bayesian Logistic Regression for Sparse Online Learning
• Online Sampling from Log-Concave Distributions
• Streaming, Distributed Variational Inference for Bayesian Nonparametrics
• Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes
• Streaming Probabilistic Deep Tensor Factorization
• Variational Auto-Regressive Gaussian Processes for Continual Learning
• Blind calibration for compressed sensing: State evolution and an online algorithm
• Quasi-Bayes properties of a recursive procedure for mixtures
• Scalable inference of topic evolution via models for latent geometric structures
• Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective
• Streaming Bayesian Inference for Crowdsourced Classification
• What is the best predictor that you can compute in five minutes using a given Bayesian hierarchical model?
• Covariance Matrices for Mean Field Variational Bayes