Scalable Bayes via Barycenter in Wasserstein Space

Abstract

Divide-and-conquer based methods for Bayesian inference provide a general approach for tractable posterior inference when the sample size is large. These methods divide the data into smaller subsets, sample from the posterior distribution of parameters in parallel on all the subsets, and combine posterior samples from all the subsets to approximate the full data posterior distribution. The smaller size of any subset compared to the full data implies that posterior sampling on any subset is computationally more efficient than sampling from the true posterior distribution. Since the combination step takes negligible time relative to sampling, posterior computations can be scaled to massive data by dividing the full data into a sufficiently large number of data subsets. One such approach relies on the geometry of posterior distributions estimated across different subsets and combines them through their barycenter in a Wasserstein space of probability measures. We provide theoretical guarantees on the accuracy of approximation that are valid in many applications. We show that the geometric method approximates the full data posterior distribution better than its competitors across diverse simulations and reproduces known results when applied to a movie ratings database.

43 pages, 7 figures, and 11 tables. The updated revision will appear in JMLR

References in corpus (3)

• Stochastic Gradient Hamiltonian Monte Carlo
• Robust and Scalable Bayes via a Median of Subset Posterior Measures
• Parallelizing MCMC via Weierstrass Sampler

Cited by in corpus (6)

• Parallel Streaming Wasserstein Barycenters
• Isometric study of Wasserstein spaces --- the real line
• Outlier-Robust Optimal Transport
• On the Wasserstein median of probability measures
• Distributed Computation for Marginal Likelihood based Model Choice
• Inference for Empirical Wasserstein Distances on Finite Spaces