Escaping the curse of dimensionality in Bayesian model based clustering
arXiv:2006.02700
Abstract
Bayesian mixture models are widely used for clustering of high-dimensional data with appropriate uncertainty quantification. However, as the dimension of the observations increases, posterior inference often tends to favor too many or too few clusters. This article explains this behavior by studying the random partition posterior in a non-standard setting with a fixed sample size and increasing data dimensionality. We provide conditions under which the finite sample posterior tends to either assign every observation to a different cluster or all observations to the same cluster as the dimension grows. Interestingly, the conditions do not depend on the choice of clustering prior, as long as all possible partitions of observations into clusters have positive prior probabilities, and hold irrespective of the true data-generating model. We then propose a class of latent mixtures for Bayesian clustering (Lamb) on a set of low-dimensional latent variables inducing a partition on the observed data. The model is amenable to scalable posterior inference and we show that it can avoid the pitfalls of high-dimensionality under mild assumptions. The proposed approach is shown to have good performance in simulation studies and an application to inferring cell types based on scRNAseq.
References in corpus (9)
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- Natural Scales in Geographical Patterns
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- High-dimensional covariance matrix estimation in approximate factor models
- Bayesian Consensus Clustering
- Discussion on Bayesian Cluster Analysis: Point Estimation and Credible Balls by Sara Wade and Zoubin Ghahramani
- Bayesian cluster analysis: Point estimation and credible balls
- Parallelizing MCMC via Weierstrass Sampler
- Generalized infinite factorization models