Algorithms of maximum likelihood data clustering with applications
arXiv:cond-mat/0204202 · doi:10.1016/S0378-4371(02)00974-3
Abstract
We address the problem of data clustering by introducing an unsupervised, parameter free approach based on maximum likelihood principle. Starting from the observation that data sets belonging to the same cluster share a common information, we construct an expression for the likelihood of any possible cluster structure. The likelihood in turn depends only on the Pearson's coefficient of the data. We discuss clustering algorithms that provide a fast and reliable approximation to maximum likelihood configurations. Compared to standard clustering methods, our approach has the advantages that i) it is parameter free, ii) the number of clusters need not be fixed in advance and iii) the interpretation of the results is transparent. In order to test our approach and compare it with standard clustering algorithms, we analyze two very different data sets: Time series of financial market returns and gene expression data. We find that different maximization algorithms produce similar cluster structures whereas the outcome of standard algorithms has a much wider variability.
Accepted by Physica A; 12 pag., 5 figures. More information at: http://www.sissa.it/dataclustering
References in corpus (1)
Cited by in corpus (17)
- Dynamics of market correlations: Taxonomy and portfolio analysis
- Dynamic asset trees and Black Monday
- Information flow between stock indices
- Systematic analysis of group identification in stock markets
- Spanning Trees and bootstrap reliability estimation in correlation based networks
- Information flow between composite stock index and individual stocks
- Emergence of time-horizon invariant correlation structure in financial returns by subtraction of the market mode
- On sampling and modeling complex systems
- Complexity analysis of the stock market
- Criticality of mostly informative samples: A Bayesian model selection approach
- Credit risk - A structural model with jumps and correlations
- Bootstrap validation of links of a minimum spanning tree
- High-speed detection of emergent market clustering via an unsupervised parallel genetic algorithm
- Identifying Dominant Industrial Sectors in Market States of the S&P 500 Financial Data
- Fast Super-Paramagnetic Clustering
- Scale-invariance of ruggedness measures in fractal fitness landscapes
- Agglomerative Likelihood Clustering