Ensuring Rapid Mixing and Low Bias for Asynchronous Gibbs Sampling

Abstract

Gibbs sampling is a Markov chain Monte Carlo technique commonly used for estimating marginal distributions. To speed up Gibbs sampling, there has recently been interest in parallelizing it by executing asynchronously. While empirical results suggest that many models can be efficiently sampled asynchronously, traditional Markov chain analysis does not apply to the asynchronous case, and thus asynchronous Gibbs sampling is poorly understood. In this paper, we derive a better understanding of the two main challenges of asynchronous Gibbs: bias and mixing time. We show experimentally that our theoretical results match practical outcomes.

References in corpus (3)

• Asynchronous Gibbs Sampling
• Rapidly Mixing Gibbs Sampling for a Class of Factor Graphs Using Hierarchy Width
• FrogWild! -- Fast PageRank Approximations on Graph Engines

Cited by in corpus (7)

• DeepBach: a Steerable Model for Bach Chorales Generation
• Streaming Graph Challenge: Stochastic Block Partition
• Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems
• Improving Gibbs Sampler Scan Quality with DoGS
• Anytime Monte Carlo
• A fast asynchronous MCMC sampler for sparse Bayesian inference
• Techniques for proving Asynchronous Convergence results for Markov Chain Monte Carlo methods