The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods

Abstract

The inverse Ising problem consists in inferring the coupling constants of an Ising model given the correlation matrix. The fastest methods for solving this problem are based on mean-field approximations, but which one performs better in the general case is still not completely clear. In the first part of this work, I summarize the formulas for several mean- field approximations and I derive new analytical expressions for the Bethe approximation, which allow to solve the inverse Ising problem without running the Susceptibility Propagation algorithm (thus avoiding the lack of convergence). In the second part, I compare the accuracy of different mean field approximations on several models (diluted ferromagnets and spin glasses) defined on random graphs and regular lattices, showing which one is in general more effective. A simple improvement over these approximations is proposed. Also a fundamental limitation is found in using methods based on TAP and Bethe approximations in presence of an external field.

v3: strongly revised version with new methods and results, 25 pages, 21 figures

References in corpus (7)

• Identification of direct residue contacts in protein-protein interaction by message passing
• High-dimensional Ising model selection using ${\ell }_1$-regularized logistic regression
• The Ising Model for Neural Data: Model Quality and Approximate Methods for Extracting Functional Connectivity
• Small-correlation expansions for the inverse Ising problem
• Bethe-Peierls approximation and the inverse Ising model
• Dynamics and Performance of Susceptibility Propagation on Synthetic Data
• Characterizing and Improving Generalized Belief Propagation Algorithms on the 2D Edwards-Anderson Model

Cited by in corpus (54)

• Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models
• A generative modeling approach for benchmarking and training shallow quantum circuits
• Data Based Identification and Prediction of Nonlinear and Complex Dynamical Systems
• Inverse statistical problems: from the inverse Ising problem to data science
• Solving Statistical Mechanics Using Variational Autoregressive Networks
• Estimation of effective temperatures in quantum annealers for sampling applications: A case study with possible applications in deep learning
• Fast pseudolikelihood maximization for direct-coupling analysis of protein structure from many homologous amino-acid sequences
• Quantum-Assisted Learning of Hardware-Embedded Probabilistic Graphical Models
• Restricted Boltzmann Machine, recent advances and mean-field theory
• Pseudolikelihood Decimation Algorithm Improving the Inference of the Interaction Network in a General Class of Ising Models
• Learning and inference in a nonequilibrium Ising model with hidden nodes
• Mapping of attention mechanisms to a generalized Potts model
• Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising
• Universal structural estimator and dynamics approximator for complex networks
• Learning Maximum Entropy Models from finite size datasets: a fast Data-Driven algorithm allows sampling from the posterior distribution
• Inverse spin glass and related maximum entropy problems
• A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
• Solving the inverse Ising problem by mean-field methods in a clustered phase space with many states
• Belief-Propagation and replicas for inference and learning in a kinetic Ising model with hidden spins
• A mean field method with correlations determined by linear response
• Detection of cheating by decimation algorithm
• Inference of the sparse kinetic Ising model using the decimation method
• Inferring effective couplings with Restricted Boltzmann Machines
• Adaptive Thouless-Anderson-Palmer approach to inverse Ising problems with quenched random fields
• A statistical physics approach to learning curves for the Inverse Ising problem
• Regularization and decimation pseudolikelihood approaches to statistical inference in $XY$-spin models
• Sparse model selection in the highly under-sampled regime
• Inferring a network from dynamical signals at its nodes
• Parallel Learning by Multitasking Neural Networks
• Resummed mean-field inference for strongly coupled data
• A Generalization of Spatial Monte Carlo Integration
• L_1-regularized Boltzmann machine learning using majorizer minimization
• Stochastic gradient method with accelerated stochastic dynamics
• Learning performance in inverse Ising problems with sparse teacher couplings
• Inference of stochastic time series with missing data
• The Cavity Master Equation: average and fixed point of the ferromagnetic model in random graphs
• Beyond inverse Ising model: structure of the analytical solution for a class of inverse problems
• Boltzmann-Machine Learning of Prior Distributions of Binarized Natural Images
• Belief propagation with multipoint correlations and its application to Inverse problem
• Inferring entropy production in many-body systems using nonequilibrium maximum entropy
• A Density Consistency approach to the inverse Ising problem
• Sparse Hopfield network reconstruction with ${\ell }_1$ regularization
• Inverse Ising inference from high-temperature re-weighting of observations
• Ising distribution as a latent variable model
• Spectral Bounds for the Ising Ferromagnet on an Arbitrary Given Graph
• Statistical Physics of Medical Diagnostics: Study of a Probabilistic Model
• Correcting beliefs in the mean-field and Bethe approximations using linear response
• The dual of the space of interactions in neural network models
• Explaining the effects of non-convergent sampling in the training of Energy-Based Models
• Empirical Bayes Method for Boltzmann Machines
• Structure Learning in Inverse Ising Problems Using ${\ell }_2$-Regularized Linear Estimator
• On the role of non-linear latent features in bipartite generative neural networks
• Solving Non-parametric Inverse Problem in Continuous Markov Random Field using Loopy Belief Propagation
• Ising Model Selection Using ${\ell }_1$-Regularized Linear Regression: A Statistical Mechanics Analysis