A new measure for assessment of clustering based on kernel density estimation

Abstract

A new clustering accuracy measure is proposed to determine the unknown number of clusters and to assess the quality of clustering of a data set given in any dimensional space. Our validity index applies the classical nonparametric univariate kernel density estimation method to the interpoint distances computed between the members of data. Being based on interpoint distances, it is free of the curse of dimensionality and therefore efficiently computable for high-dimensional situations where the number of study variables can be larger than the sample size. The proposed measure is compatible with any clustering algorithm and with every kind of data set where the interpoint distance measure can be defined to have a density function. Simulation study proves its superiority over widely used cluster validity indices like the average silhouette width and the Dunn index, whereas its applicability is shown with respect to a high-dimensional Biostatistical study of Alon data set and a large Astrostatistical application of time series with light curves of new variable stars.

arXiv comment16 pages

References in corpus (9)

• Ward's Hierarchical Clustering Method: Clustering Criterion and Agglomerative Algorithm
• Unsupervised classification of eclipsing binary light curves through k-medoids clustering
• Distinction of groups of gamma-ray bursts in the BATSE catalog through fuzzy clustering
• Clustering of Gamma-Ray bursts through kernel principal component analysis
• Two phase formation of massive elliptical galaxies : study through cross-correlation including spatial effect
• A new nonparametric test for two sample multivariate location problem with application to astronomy
• Clustering of eclipsing binary light curves through functional principal component analysis
• A new nonparametric interpoint distance-based measure for assessment of clustering
• 1318 New Variable Stars in a 0.25 Square Degree Region of the Galactic Plane

Cited by in corpus (6)

• Clustering of eclipsing binary light curves through functional principal component analysis
• A new nonparametric interpoint distance-based measure for assessment of clustering
• A new interpoint distance-based clustering algorithm using kernel density estimation
• Evaluation of the number of clusters in a data set using $p$-values from Multiple Tests of Hypotheses
• On the Use of Relative Validity Indices for Comparing Clustering Approaches
• Confirmation of Binary Clustering in Gamma-Ray Bursts through an Integrated $p$-value from Multiple Nonparametric Tests of Hypotheses