Agglomerative Likelihood Clustering
arXiv:1908.00951 · doi:10.1088/1742-5468/ac3661
Abstract
We consider the problem of fast time-series data clustering. Building on previous work modeling the correlation-based Hamiltonian of spin variables we present an updated fast non-expensive Agglomerative Likelihood Clustering algorithm (ALC). The method replaces the optimized genetic algorithm based approach (f-SPC) with an agglomerative recursive merging framework inspired by previous work in Econophysics and Community Detection. The method is tested on noisy synthetic correlated time-series data-sets with built-in cluster structure to demonstrate that the algorithm produces meaningful non-trivial results. We apply it to time-series data-sets as large as 20,000 assets and we argue that ALC can reduce compute time costs and resource usage cost for large scale clustering for time-series applications while being serialized, and hence has no obvious parallelization requirement. The algorithm can be an effective choice for state-detection for online learning in a fast non-linear data environment because the algorithm requires no prior information about the number of clusters.
arXiv comment15 pages, 8 figures
References in corpus (17)
- Fast unfolding of communities in large networks
- Inverse Cubic Law for the Probability Distribution of Stock Price Variations
- Correlation, hierarchies, and networks in financial markets
- Cleaning large correlation matrices: tools from random matrix theory
- Local resolution-limit-free Potts model for community detection
- Mutual Information Rate-Based Networks in Financial Markets
- Faster unfolding of communities: speeding up the Louvain algorithm
- A model for correlations in stock markets
- An analysis of Cross-correlations in South African Market data
- On the criticality of inferred models
- Data clustering and noise undressing for correlation matrices
- Hierarchically nested factor model from multivariate data
- Algorithms of maximum likelihood data clustering with applications
- Detecting intraday financial market states using temporal clustering
- Hierarchical causality in financial economics
- High-speed detection of emergent market clustering via an unsupervised parallel genetic algorithm
- Fast Super-Paramagnetic Clustering