A simple example of Dirichlet process mixture inconsistency for the number of components

Abstract

For data assumed to come from a finite mixture with an unknown number of components, it has become common to use Dirichlet process mixtures (DPMs) not only for density estimation, but also for inferences about the number of components. The typical approach is to use the posterior distribution on the number of components occurring so far --- that is, the posterior on the number of clusters in the observed data. However, it turns out that this posterior is not consistent --- it does not converge to the true number of components. In this note, we give an elementary demonstration of this inconsistency in what is perhaps the simplest possible setting: a DPM with normal components of unit variance, applied to data from a "mixture" with one standard normal component. Further, we find that this example exhibits severe inconsistency: instead of going to 1, the posterior probability that there is one cluster goes to 0.

Cited by in corpus (27)

• Bayesian Consensus Clustering
• Bayesian cluster analysis: Point estimation and credible balls
• Discussion on Bayesian Cluster Analysis: Point Estimation and Credible Balls by Sara Wade and Zoubin Ghahramani
• Generalized mixtures of finite mixtures and telescoping sampling
• Infinite Mixtures of Infinite Factor Analysers
• Clustering consistency with Dirichlet process mixtures
• ClusterCluster: Parallel Markov Chain Monte Carlo for Dirichlet Process Mixtures
• Bayesian Group Learning for Shot Selection of Professional Basketball Players
• Heterogeneous Regression Models for Clusters of Spatial Dependent Data
• A marginal sampler for $\sigma$-Stable Poisson-Kingman mixture models
• Finite mixture models do not reliably learn the number of components
• Bayesian Spatial Homogeneity Pursuit for Survival Data with an Application to the SEER Respiratory Cancer Data
• A Tractable Fully Bayesian Method for the Stochastic Block Model
• Spatial Clustering Regression of Count Value Data via Bayesian Mixture of Finite Mixtures
• Markov Switching
• Heterogeneity Pursuit for Spatial Point Pattern with Application to Tree Locations: A Bayesian Semiparametric Recourse
• Distributed, partially collapsed MCMC for Bayesian Nonparametrics
• Optimal Bayesian estimation of Gaussian mixtures with growing number of components
• Streaming, Distributed Variational Inference for Bayesian Nonparametrics
• Nested Dirichlet models for unsupervised attack pattern detection in honeypot data
• Posterior Distribution for the Number of Clusters in Dirichlet Process Mixture Models
• Infinite mixtures of multivariate normal-inverse Gaussian distributions for clustering of skewed data
• Accelerated Parallel Non-conjugate Sampling for Bayesian Non-parametric Models
• Heterogeneity Learning for SIRS model: an Application to the COVID-19
• Few-shot time series segmentation using prototype-defined infinite hidden Markov models
• Controlling for sparsity in sparse factor analysis models: adaptive latent feature sharing for piecewise linear dimensionality reduction
• Semi- and Weakly-supervised Human Pose Estimation