Information based clustering

Abstract

In an age of increasingly large data sets, investigators in many different disciplines have turned to clustering as a tool for data analysis and exploration. Existing clustering methods, however, typically depend on several nontrivial assumptions about the structure of data. Here we reformulate the clustering problem from an information theoretic perspective which avoids many of these assumptions. In particular, our formulation obviates the need for defining a cluster "prototype", does not require an a priori similarity metric, is invariant to changes in the representation of the data, and naturally captures non-linear relations. We apply this approach to different domains and find that it consistently produces clusters that are more coherent than those extracted by existing algorithms. Finally, our approach provides a way of clustering based on collective notions of similarity rather than the traditional pairwise measures.

To appear in Proceedings of the National Academy of Sciences USA, 11 pages, 9 figures

References in corpus (2)

• Network information and connected correlations
• Information based clustering

Cited by in corpus (28)

• Correlation, hierarchies, and networks in financial markets
• Information flow and optimization in transcriptional control
• Inverse statistical problems: from the inverse Ising problem to data science
• Information based clustering
• Evolution of bow-tie architectures in biology
• Statistical mechanics for metabolic networks during steady-state growth
• A functional clustering algorithm for the analysis of dynamic network data
• Hierarchically nested factor model from multivariate data
• Topological Information Data Analysis
• Distributed Information-Theoretic Clustering
• Deep Semi-Supervised Anomaly Detection
• The Convex Information Bottleneck Lagrangian
• Population stratification using a statistical model on hypergraphs
• A unified data representation theory for network visualization, ordering and coarse-graining
• AMIC: An Adaptive Information Theoretic Method to Identify Multi-Scale Temporal Correlations in Big Time Series Data -- Accepted Version
• Demystifying Information-Theoretic Clustering
• Energy-Based Clustering: Fast and Robust Clustering of Data with Known Likelihood Functions
• A framework for studying behavioral evolution by reconstructing ancestral repertoires
• Mining the modular structure of protein interaction networks
• Significance analysis and statistical mechanics: an application to clustering
• What if we had no Wikipedia? Domain-independent Term Extraction from a Large News Corpus
• Bregman Divergence Bounds and Universality Properties of the Logarithmic Loss
• Improving Quality of Hierarchical Clustering for Large Data Series
• Estimation of mutual information via quantum kernel method
• Multilinear objective function-based clustering
• Information based clustering: Supplementary material
• Information Theory of Genomes
• Covariance-based Dissimilarity Measures Applied to Clustering Wide-sense Stationary Ergodic Processes