Accelerated Parallel Non-conjugate Sampling for Bayesian Non-parametric Models

Abstract

Inference of latent feature models in the Bayesian nonparametric setting is generally difficult, especially in high dimensional settings, because it usually requires proposing features from some prior distribution. In special cases, where the integration is tractable, we can sample new feature assignments according to a predictive likelihood. We present a novel method to accelerate the mixing of latent variable model inference by proposing feature locations based on the data, as opposed to the prior. First, we introduce an accelerated feature proposal mechanism that we show is a valid MCMC algorithm for posterior inference. Next, we propose an approximate inference strategy to perform accelerated inference in parallel. A two-stage algorithm that combines the two approaches provides a computationally attractive method that can quickly reach local convergence to the posterior distribution of our model, while allowing us to exploit parallelization.

arXiv commentTo appear in Statistics & Computing

References in corpus (7)

• Auto-Encoding Variational Bayes
• Philosophy and the practice of Bayesian statistics
• Elliptical slice sampling
• Joint modeling of multiple time series via the beta process with application to motion capture segmentation
• A simple example of Dirichlet process mixture inconsistency for the number of components
• MAD-Bayes: MAP-based Asymptotic Derivations from Bayes
• ClusterCluster: Parallel Markov Chain Monte Carlo for Dirichlet Process Mixtures