Unsupervised hierarchical clustering using the learning dynamics of RBMs

Abstract

Datasets in the real world are often complex and to some degree hierarchical, with groups and sub-groups of data sharing common characteristics at different levels of abstraction. Understanding and uncovering the hidden structure of these datasets is an important task that has many practical applications. To address this challenge, we present a new and general method for building relational data trees by exploiting the learning dynamics of the Restricted Boltzmann Machine (RBM). Our method is based on the mean-field approach, derived from the Plefka expansion, and developed in the context of disordered systems. It is designed to be easily interpretable. We tested our method in an artificially created hierarchical dataset and on three different real-world datasets (images of digits, mutations in the human genome, and a homologous family of proteins). The method is able to automatically identify the hierarchical structure of the data. This could be useful in the study of homologous protein sequences, where the relationships between proteins are critical for understanding their function and evolution.

arXiv commentVersion accepted in Physical Review E

References in corpus (10)

• Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
• Identification of direct residue contacts in protein-protein interaction by message passing
• Inverse Statistical Physics of Protein Sequences: A Key Issues Review
• Emergence of Compositional Representations in Restricted Boltzmann Machines
• Thermodynamics of Restricted Boltzmann Machines and related learning dynamics
• Spectral Dynamics of Learning Restricted Boltzmann Machines
• High-temperature Expansions and Message Passing Algorithms
• A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
• Learning a Restricted Boltzmann Machine using biased Monte Carlo sampling
• Cascade of Phase Transitions for Multi-Scale Clustering

Cited by in corpus (8)

• Inferring effective couplings with Restricted Boltzmann Machines
• The unbearable lightness of Restricted Boltzmann Machines: Theoretical Insights and Biological Applications
• Interpretable machine learning of amino acid patterns in proteins: a statistical ensemble approach
• Inferring Higher-Order Couplings with Neural Networks
• Hyperspectral image segmentation with a machine learning model trained using quantum annealer
• Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings
• Ultrametric identities in glassy models of Natural Evolution
• Emergence of Nonequilibrium Latent Cycles in Unsupervised Generative Modeling