Generalized mixtures of finite mixtures and telescoping sampling

Abstract

Within a Bayesian framework, a comprehensive investigation of mixtures of finite mixtures (MFMs), i.e., finite mixtures with a prior on the number of components, is performed. This model class has applications in model-based clustering as well as for semi-parametric density estimation and requires suitable prior specifications and inference methods to exploit its full potential. We contribute by considering a generalized class of MFMs where the hyperparameter ${\gamma }_K$ of a symmetric Dirichlet prior on the weight distribution depends on the number of components. We show that this model class may be regarded as a Bayesian non-parametric mixture outside the class of Gibbs-type priors. We emphasize the distinction between the number of components $K$ of a mixture and the number of clusters $K_{+}$, i.e., the number of filled components given the data. In the MFM model, $K_{+}$ is a random variable and its prior depends on the prior on $K$ and on the hyperparameter ${\gamma }_K$. We employ a flexible prior distribution for the number of components $K$ and derive the corresponding prior on the number of clusters $K_{+}$ for generalized MFMs. For posterior inference, we propose the novel telescoping sampler which allows Bayesian inference for mixtures with arbitrary component distributions without resorting to reversible jump Markov chain Monte Carlo (MCMC) methods. The telescoping sampler explicitly samples the number of components, but otherwise requires only the usual MCMC steps of a finite mixture model. The ease of its application using different component distributions is demonstrated on several data sets.

References in corpus (6)

• Model-based clustering based on sparse finite Gaussian mixtures
• Are Gibbs-type priors the most natural generalization of the Dirichlet process?
• A simple example of Dirichlet process mixture inconsistency for the number of components
• Spying on the prior of the number of data clusters and the partition distribution in Bayesian cluster analysis
• A new parametrization of the Gnedin-Fisher species sampling model
• How many data clusters are in the Galaxy data set? Bayesian cluster analysis in action

Cited by in corpus (7)

• Spying on the prior of the number of data clusters and the partition distribution in Bayesian cluster analysis
• Bayesian inference for continuous-time hidden Markov models with an unknown number of states
• Bayesian clustering of multiple zero-inflated outcomes
• Model based clustering of multinomial count data
• Bayesian mixture models (in)consistency for the number of clusters
• How many data clusters are in the Galaxy data set? Bayesian cluster analysis in action
• Bayesian Finite Mixture Models