Model-based clustering based on sparse finite Gaussian mixtures

Abstract

In the framework of Bayesian model-based clustering based on a finite mixture of Gaussian distributions, we present a joint approach to estimate the number of mixture components and identify cluster-relevant variables simultaneously as well as to obtain an identified model. Our approach consists in specifying sparse hierarchical priors on the mixture weights and component means. In a deliberately overfitting mixture model the sparse prior on the weights empties superfluous components during MCMC. A straightforward estimator for the true number of components is given by the most frequent number of non-empty components visited during MCMC sampling. Specifying a shrinkage prior, namely the normal gamma prior, on the component means leads to improved parameter estimates as well as identification of cluster-relevant variables. After estimating the mixture model using MCMC methods based on data augmentation and Gibbs sampling, an identified model is obtained by relabeling the MCMC output in the point process representation of the draws. This is performed using $K$-centroids cluster analysis based on the Mahalanobis distance. We evaluate our proposed strategy in a simulation setup with artificial data and by applying it to benchmark data sets.

22 pages

References in corpus (3)

• Generalized Beta Mixtures of Gaussians
• On the posterior distribution of the number of components in a finite mixture
• PReMiuM: An R Package for Profile Regression Mixture Models using Dirichlet Processes

Cited by in corpus (24)

• Variable Selection Methods for Model-based Clustering
• Variable Selection for Latent Class Analysis with Application to Low Back Pain Diagnosis
• Identifying Mixtures of Mixtures Using Bayesian Estimation
• Significance Mode Analysis (SigMA) for hierarchical structures. An application to the Sco-Cen OB association
• Generalized mixtures of finite mixtures and telescoping sampling
• Multiple-Speaker Localization Based on Direct-Path Features and Likelihood Maximization with Spatial Sparsity Regularization
• On the identifiability of Bayesian factor analytic models
• Reducibility and Statistical-Computational Gaps from Secret Leakage
• Clustering high dimensional mixed data to uncover sub-phenotypes:joint analysis of phenotypic and genotypic data
• Robust Clustering with Subpopulation-specific Deviations
• Finite mixture models do not reliably learn the number of components
• Clustering Multivariate Data using Factor Analytic Bayesian Mixtures with an Unknown Number of Components
• Bayesian clustering of multiple zero-inflated outcomes
• A Two-Stage Bayesian Semiparametric Model for Novelty Detection with Robust Prior Information
• Estimation and selection for high-order Markov chains with Bayesian mixture transition distribution models
• Identifying Brexit voting patterns in the British House of Commons: an analysis based on Bayesian mixture models with flexible concomitant covariate effects
• How many data clusters are in the Galaxy data set? Bayesian cluster analysis in action
• Model based clustering of multinomial count data
• Bayesian mixture models (in)consistency for the number of clusters
• Variational Markov chain mixtures with automatic component selection
• Bayesian Finite Mixture Models
• A Bayesian panel VAR model to analyze the impact of climate change on high-income economies
• Bayesian shrinkage in mixture of experts models: Identifying robust determinants of class membership
• Mixtures of multivariate generalized linear models with overlapping clusters