Inferring effective couplings with Restricted Boltzmann Machines

Abstract

Generative models offer a direct way of modeling complex data. Energy-based models attempt to encode the statistical correlations observed in the data at the level of the Boltzmann weight associated with an energy function in the form of a neural network. We address here the challenge of understanding the physical interpretation of such models. In this study, we propose a simple solution by implementing a direct mapping between the Restricted Boltzmann Machine and an effective Ising spin Hamiltonian. This mapping includes interactions of all possible orders, going beyond the conventional pairwise interactions typically considered in the inverse Ising (or Boltzmann Machine) approach, and allowing the description of complex datasets. Earlier works attempted to achieve this goal, but the proposed mappings were inaccurate for inference applications, did not properly treat the complexity of the problem, or did not provide precise prescriptions for practical application. To validate our method, we performed several controlled inverse numerical experiments in which we trained the RBMs using equilibrium samples of predefined models with local external fields, 2-body and 3-body interactions in different sparse topologies. The results demonstrate the effectiveness of our proposed approach in learning the correct interaction network and pave the way for its application in modeling interesting binary variable datasets. We also evaluate the quality of the inferred model based on different training methods.

arXiv comment17 figures, 39 pages

References in corpus (23)

• Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
• Searching for collective behavior in a network of real neurons
• Inverse statistical problems: from the inverse Ising problem to data science
• A Theory of Generative ConvNet
• Emergence of Compositional Representations in Restricted Boltzmann Machines
• Statistical Genetics and Evolution of Quantitative Traits
• Annealed Importance Sampling
• The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods
• Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors
• Thermodynamics of Restricted Boltzmann Machines and related learning dynamics
• A basic introduction to large deviations: Theory, applications, simulations
• Spectral Dynamics of Learning Restricted Boltzmann Machines
• Machine learning determination of dynamical parameters: The Ising model case
• Bethe-Peierls approximation and the inverse Ising model
• Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines
• Exact representations of many body interactions with RBM neural networks
• Unsupervised hierarchical clustering using the learning dynamics of RBMs
• A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
• Learning a Restricted Boltzmann Machine using biased Monte Carlo sampling
• A new test for random number generators: Schwinger-Dyson equations for the Ising model
• Inferring genetic fitness from genomic data
• A Density Consistency approach to the inverse Ising problem
• A statistical approach to topological entanglement: Boltzmann machine representation of high-order irreducible correlation

Cited by in corpus (7)

• The unbearable lightness of Restricted Boltzmann Machines: Theoretical Insights and Biological Applications
• Unification of Symmetries Inside Neural Networks: Transformer, Feedforward and Neural ODE
• The effect of priors on Learning with Restricted Boltzmann Machines
• Inferring Higher-Order Couplings with Neural Networks
• On the role of non-linear latent features in bipartite generative neural networks
• Global thermodynamic manifold for conservative control of stochastic systems
• Neural quantum states for entanglement depth certification from randomized Pauli measurements