A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process

Abstract

The hierarchical Dirichlet process (HDP) has become an important Bayesian nonparametric model for grouped data, such as document collections. The HDP is used to construct a flexible mixed-membership model where the number of components is determined by the data. As for most Bayesian nonparametric models, exact posterior inference is intractable---practitioners use Markov chain Monte Carlo (MCMC) or variational inference. Inspired by the split-merge MCMC algorithm for the Dirichlet process (DP) mixture model, we describe a novel split-merge MCMC sampling algorithm for posterior inference in the HDP. We study its properties on both synthetic data and text corpora. We find that split-merge MCMC for the HDP can provide significant improvements over traditional Gibbs sampling, and we give some understanding of the data properties that give rise to larger improvements.

References in corpus (1)

• Distance Dependent Chinese Restaurant Processes

Cited by in corpus (7)

• Joint modeling of multiple time series via the beta process with application to motion capture segmentation
• Learning with fuzzy hypergraphs: a topical approach to query-oriented text summarization
• Online Bayesian Passive-Aggressive Learning
• Blocked Gibbs sampler for hierarchical Dirichlet processes
• Scaling-up Split-Merge MCMC with Locality Sensitive Sampling (LSS)
• Nonparametric Deconvolution Models
• UoB at SemEval-2020 Task 1: Automatic Identification of Novel Word Senses