Inferring Higher-Order Couplings with Neural Networks

Abstract

Maximum entropy methods, rooted in the inverse Ising/Potts problem from statistical physics, are widely used to model pairwise interactions in complex systems across disciplines such as bioinformatics and neuroscience. While successful, these approaches often fail to capture higher-order interactions that are critical for understanding collective behavior. In contrast, modern machine learning methods can model such interactions, but their interpretability often comes at a prohibitive computational cost. Restricted Boltzmann Machines (RBMs) provide a computationally efficient alternative by encoding statistical correlations through hidden units in a bipartite architecture. In this work, we introduce a method that maps RBMs onto generalized Potts models, enabling the systematic extraction of interactions up to arbitrary order. Leveraging large-$N$ approximations, made tractable by the RBM's structure, we extract effective many-body couplings with minimal computational effort. We further propose a robust framework for recovering higher-order interactions in more complex generative models, and introduce a simple gauge-fixing scheme for the effective Potts representation. Validation on synthetic data demonstrates accurate recovery of two- and three-body interactions. Applied to protein sequence data, our method reconstructs contact maps with high fidelity and outperforms state-of-the-art inverse Potts models. These results establish RBMs as a powerful and efficient tool for modeling higher-order structure in high-dimensional categorical data.

24 Pages and 9 Figures

References in corpus (20)

• Weak pairwise correlations imply strongly correlated network states in a neural population
• Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
• Identification of direct residue contacts in protein-protein interaction by message passing
• The physics of higher-order interactions in complex systems
• Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models
• Information flow and optimization in transcriptional control
• Inverse statistical problems: from the inverse Ising problem to data science
• Inverse Statistical Physics of Protein Sequences: A Key Issues Review
• Fast pseudolikelihood maximization for direct-coupling analysis of protein structure from many homologous amino-acid sequences
• Pairwise maximum entropy models for studying large biological systems: when they can and when they can't work
• Collective behavior of place and non-place neurons in the hippocampal network
• Statistical physics of pairwise probability models
• Efficient generative modeling of protein sequences using simple autoregressive models
• Improving contact prediction along three dimensions
• Machine learning determination of dynamical parameters: The Ising model case
• Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines
• Unsupervised hierarchical clustering using the learning dynamics of RBMs
• Higher-order interactions in statistical physics and machine learning: A model-independent solution to the inverse problem at equilibrium
• Inferring effective couplings with Restricted Boltzmann Machines
• The unbearable lightness of Restricted Boltzmann Machines: Theoretical Insights and Biological Applications

Cited by in corpus (1)

• On the role of non-linear latent features in bipartite generative neural networks