Infinite Mixtures of Infinite Factor Analysers

Abstract

Factor-analytic Gaussian mixture models are often employed as a model-based approach to clustering high-dimensional data. Typically, the numbers of clusters and latent factors must be specified in advance of model fitting, and remain fixed. The pair which optimises some model selection criterion is then chosen. For computational reasons, models in which the number of latent factors differ across clusters are rarely considered. Here the infinite mixture of infinite factor analysers (IMIFA) model is introduced. IMIFA employs a Pitman-Yor process prior to facilitate automatic inference of the number of clusters using the stick-breaking construction and a slice sampler. Furthermore, IMIFA employs multiplicative gamma process shrinkage priors to allow cluster-specific numbers of factors, automatically inferred via an adaptive Gibbs sampler. IMIFA is presented as the flagship of a family of factor-analytic mixture models, providing flexible approaches to clustering high-dimensional data. Applications to a benchmark data set, metabolomic spectral data, and a manifold learning handwritten digit example illustrate the IMIFA model and its advantageous features. These include obviating the need for model selection criteria, reducing the computational burden associated with the search of the model space, improving clustering performance by allowing cluster-specific numbers of factors, and quantifying uncertainty in the numbers of clusters and cluster-specific factors.

arXiv commentPublished in Bayesian Analysis

References in corpus (10)

• Statistical analysis of factor models of high dimension
• Are Gibbs-type priors the most natural generalization of the Dirichlet process?
• Nonparametric Bayesian sparse factor models with application to gene expression modeling
• A simple example of Dirichlet process mixture inconsistency for the number of components
• Overfitting Bayesian Mixture Models with an Unknown Number of Components
• Inconsistency of Pitman-Yor process mixtures for the number of components
• A note on the multiplicative gamma process
• Clustering South African households based on their asset status using latent variable models
• Infinite Mixtures of Infinite Factor Analysers
• Overfitting Bayesian Mixtures of Factor Analyzers with an Unknown Number of Components

Cited by in corpus (7)

• Infinite Mixtures of Infinite Factor Analysers
• On the identifiability of Bayesian factor analytic models
• Generalized Cumulative Shrinkage Process Priors with Applications to Sparse Bayesian Factor Analysis
• A Latent Shrinkage Position Model for Binary and Count Network Data
• Identifying Brexit voting patterns in the British House of Commons: an analysis based on Bayesian mixture models with flexible concomitant covariate effects
• Partial membership models for soft clustering of multivariate football player performance data
• Mixtures of multivariate generalized linear models with overlapping clusters