Recovering the number of clusters in data sets with noise features using feature rescaling factors

Abstract

In this paper we introduce three methods for re-scaling data sets aiming at improving the likelihood of clustering validity indexes to return the true number of spherical Gaussian clusters with additional noise features. Our method obtains feature re-scaling factors taking into account the structure of a given data set and the intuitive idea that different features may have different degrees of relevance at different clusters. We experiment with the Silhouette (using squared Euclidean, Manhattan, and the p${}^{th}$ power of the Minkowski distance), Dunn's, Calinski-Harabasz and Hartigan indexes on data sets with spherical Gaussian clusters with and without noise features. We conclude that our methods indeed increase the chances of estimating the true number of clusters in a data set.

Cited by in corpus (22)

• CNAK : Cluster Number Assisted K-means
• Morphological Neuron Classification Using Machine Learning
• A-Ward_p\b{eta}: Effective hierarchical clustering using the Minkowski metric and a fast k -means initialisation
• A scalable saliency-based Feature selection method with instance level information
• Assisted Requirements Selection by Clustering
• CRAD: Clustering with Robust Autocuts and Depth
• Temporal Clustering in Dynamic Networks with Tensor Decomposition
• Deep Learning of Phase Transitions for Quantum Spin Chains from Correlation Aspects
• Robust statistical tools for identifying multiple stellar populations in globular clusters in the presence of measurement errors. A case study: NGC 2808
• Magnetism in four-layered Aurivillius Bi${}_5$FeTi${}_3$O${}_{15}$ at high pressures : A nuclear forward scattering study
• Dual-Triplet Metric Learning for Unsupervised Domain Adaptation in Video-Based Face Recognition
• Datacube segmentation via Deep Spectral Clustering
• Semi-Supervised Learning Approach to Discover Enterprise User Insights from Feedback and Support
• Data Driven Aircraft Trajectory Prediction with Deep Imitation Learning
• Semi-Supervised Class Discovery
• Optimal Number of Choices in Rating Contexts
• Deep learning of topological phase transitions from entanglement aspects: An unsupervised way
• *K-means and Cluster Models for Cancer Signatures
• Penalized k-means algorithms for finding the correct number of clusters in a dataset
• Probabilistic Performance-Pattern Decomposition (PPPD): analysis framework and applications to stochastic mechanical systems
• Detecting Biological Locomotion in Video: A Computational Approach
• Statistical Industry Classification