Fast model-based clustering of partial records

Abstract

Partially recorded data are frequently encountered in many applications and usually clustered by first removing incomplete cases or features with missing values, or by imputing missing values, followed by application of a clustering algorithm to the resulting altered dataset. Here, we develop clustering methodology through a model-based approach using the marginal density for the observed values, assuming a finite mixture model of multivariate $t$ distributions. We compare our approximate algorithm to the corresponding full expectation-maximization (EM) approach that considers the missing values in the incomplete data set and makes a missing at random (MAR) assumption, as well as case deletion and imputation methods. Since only the observed values are utilized, our approach is computationally more efficient than imputation or full EM. Simulation studies demonstrate that our approach has favorable recovery of the true cluster partition compared to case deletion and imputation under various missingness mechanisms, and is at least competitive with the full EM approach, even when MAR assumptions are violated. Our methodology is demonstrated on a problem of clustering gamma-ray bursts and is implemented at https://github.com/emilygoren/MixtClust.

arXiv comment18 pages, 8 figures, 1 table

References in corpus (16)

• Natural Scales in Geographical Patterns
• The Physics of Gamma-Ray Bursts
• Three types of gamma-ray bursts
• Fermi-LAT Observations of the Gamma-ray Burst GRB 130427A
• The ultra-long Gamma-Ray Burst 111209A: the collapse of a blue supergiant?
• Classification and clustering of sequencing data using a Poisson model
• $k$-POD: A Method for $k$-Means Clustering of Missing Data
• Statistical Evidence for Three classes of Gamma-ray Bursts
• Short vs. Long Gamma-Ray Bursts: A Comprehensive Study of Energetics and Prompt Gamma-Ray Correlations
• A Re-defined and Generalized Percent-Overlap-of-Activation Measure for Studies of fMRI Reproducibility and its Use in Identifying Outlier Activation Maps
• Gaussian-Mixture-Model-based Cluster Analysis Finds Five Kinds of Gamma Ray Bursts in the BATSE Catalog
• Gaussian-mixture-model-based cluster analysis of gamma-ray bursts in the BATSE catalog
• Mixtures of Generalized Hyperbolic Distributions and Mixtures of Skew-t Distributions for Model-Based Clustering with Incomplete Data
• An efficient $k$-means-type algorithm for clustering datasets with incomplete records
• Kernel-estimated Nonparametric Overlap-Based Syncytial Clustering
• Efficient Bandwidth Estimation in Two-dimensional Filtered Backprojection Reconstruction

Cited by in corpus (2)

• Latent characterisation of the complete BATSE gamma ray bursts catalogue using Gaussian mixture of factor analysers and model-estimated overlap-based syncytial clustering
• Multi-layered characterization of hot stellar systems with confidence