Detecting Hierarchical Clusters and Estimating their Modularity Directly from Dendrograms

Abstract

Identifying possible clusters in datasets and estimating their hierarchical modularity are central tasks in pattern recognition. In the present work, concepts and methodologies are described for performing these tasks while considering only the density of mergings obtained from hierarchical representations (dendrograms) of data inter-relationship along a scale variable. More specifically, the mergings of subclusters along the scale variable are obtained, yielding a respective merging density function. After this function is equalized along the scale variable, peak detection is applied in order to estimate, within a specified resolution, the main hierarchical levels and their clusters. After quantifying infinitesimal modularity of the dendrogram at a fixed scale value, taking into account the uniformity of the size of the identified clusters and their average size, the overall, average, and group hierarchical modularities are obtained. The potential of the reported approach is illustrated for some types of data and dendrograms, and the possibility of recursive cluster detection is also considered.

24 pages and 16 figures