Overfitting Bayesian Mixture Models with an Unknown Number of Components

Abstract

This paper proposes solutions to three issues pertaining to the estimation of finite mixture models with an unknown number of components: the non-identifiability induced by overfitting the number of components, the mixing limitations of standard Markov Chain Monte Carlo (MCMC) sampling techniques, and the related label switching problem. An overfitting approach is used to estimate the number of components in a finite mixture model via a Zmix algorithm. Zmix provides a bridge between multidimensional samplers and test based estimation methods, whereby priors are chosen to encourage extra groups to have weights approaching zero. MCMC sampling is made possible by the implementation of prior parallel tempering, an extension of parallel tempering. Zmix can accurately estimate the number of components, posterior parameter estimates and allocation probabilities given a sufficiently large sample size. The results will reflect uncertainty in the final model and will report the range of possible candidate models and their respective estimated probabilities from a single run. Label switching is resolved with a computationally light-weight method, Zswitch, developed for overfitted mixtures by exploiting the intuitiveness of allocation-based relabelling algorithms and the precision of label-invariant loss functions. Four simulation studies are included to illustrate Zmix and Zswitch, as well as three case studies from the literature. All methods are available as part of the R package Zmix, which can currently be applied to univariate Gaussian mixture models

References in corpus (1)

• Bayesian finite mixtures: a note on prior specification and posterior computation

Cited by in corpus (12)

• Infinite Mixtures of Infinite Factor Analysers
• Clustering high dimensional mixed data to uncover sub-phenotypes:joint analysis of phenotypic and genotypic data
• Robust Clustering with Subpopulation-specific Deviations
• Overfitting Bayesian Mixtures of Factor Analyzers with an Unknown Number of Components
• Clustering Multivariate Data using Factor Analytic Bayesian Mixtures with an Unknown Number of Components
• A survey on Bayesian inference for Gaussian mixture model
• Sharp Analysis of Expectation-Maximization for Weakly Identifiable Models
• Fast sampling and model selection for Bayesian mixture models
• Overfitting hidden Markov models with an unknown number of states
• Model based clustering of multinomial count data
• Supervised Robust Profile Clustering
• An Imputation model by Dirichlet Process Mixture of Elliptical Copulas for Data of Mixed Type