Bayesian Conditional Density Filtering

Abstract

We propose a Conditional Density Filtering (C-DF) algorithm for efficient online Bayesian inference. C-DF adapts MCMC sampling to the online setting, sampling from approximations to conditional posterior distributions obtained by propagating surrogate conditional sufficient statistics (a function of data and parameter estimates) as new data arrive. These quantities eliminate the need to store or process the entire dataset simultaneously and offer a number of desirable features. Often, these include a reduction in memory requirements and runtime and improved mixing, along with state-of-the-art parameter inference and prediction. These improvements are demonstrated through several illustrative examples including an application to high dimensional compressed regression. Finally, we show that C-DF samples converge to the target posterior distribution asymptotically as sampling proceeds and more data arrives.

41 pages, 7 figures, 12 tables

References in corpus (9)

• Expectation Propagation for approximate Bayesian inference
• Tractable Inference for Complex Stochastic Processes
• Speeding Up MCMC by Efficient Data Subsampling
• Robust and Scalable Bayes via a Median of Subset Posterior Measures
• Zero-Truncated Poisson Tensor Factorization for Massive Binary Tensors
• Consistency and fluctuations for stochastic gradient Langevin dynamics
• Scalable Bayesian Non-Negative Tensor Factorization for Massive Count Data
• Virtual Vector Machine for Bayesian Online Classification
• Bayesian Compressed Regression

Cited by in corpus (4)

• Scalable Bayesian Non-Negative Tensor Factorization for Massive Count Data
• Big Learning with Bayesian Methods
• Streaming Gibbs Sampling for LDA Model
• High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models