A statistical physics approach to learning curves for the Inverse Ising problem

Abstract

Using methods of statistical physics, we analyse the error of learning couplings in large Ising models from independent data (the inverse Ising problem). We concentrate on learning based on local cost functions, such as the pseudo-likelihood method for which the couplings are inferred independently for each spin. Assuming that the data are generated from a true Ising model, we compute the reconstruction error of the couplings using a combination of the replica method with the cavity approach for densely connected systems. We show that an explicit estimator based on a quadratic cost function achieves minimal reconstruction error, but requires the length of the true coupling vector as prior knowledge. A simple mean field estimator of the couplings which does not need such knowledge is asymptotically optimal, i.e. when the number of observations is much large than the number of spins. Comparison of the theory with numerical simulations shows excellent agreement for data generated from two models with random couplings in the high temperature region: a model with independent couplings (Sherrington-Kirkpatrick model), and a model where the matrix of couplings has a Wishart distribution.

References in corpus (19)

• Weak pairwise correlations imply strongly correlated network states in a neural population
• Identification of direct residue contacts in protein-protein interaction by message passing
• The Bethe lattice spin glass revisited
• The Ising Model for Neural Data: Model Quality and Approximate Methods for Extracting Functional Connectivity
• Inverse Ising inference using all the data
• Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data
• Statistical physics of pairwise probability models
• Small-correlation expansions for the inverse Ising problem
• Mean-Field Equations for Spin Models with Orthogonal Interaction Matrices
• The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods
• Pseudolikelihood Decimation Algorithm Improving the Inference of the Interaction Network in a General Class of Ising Models
• A Theory of Solving TAP Equations for Ising Models with General Invariant Random Matrices
• Faster solutions of the inverse pairwise Ising problem
• Bethe-Peierls approximation and the inverse Ising model
• Consistent inference of a general model using the pseudo-likelihood method
• Statistical Mechanics of High-Dimensional Inference
• Learning of couplings for random asymmetric kinetic Ising models revisited: random correlation matrices and learning curves
• Regularization and decimation pseudolikelihood approaches to statistical inference in $XY$-spin models
• Learning a spin glass: determining Hamiltonians from metastable states

Cited by in corpus (5)

• Direct Coupling Analysis of Epistasis in Allosteric Materials
• Inverse Ising problem in continuous time: A latent variable approach
• Learning performance in inverse Ising problems with sparse teacher couplings
• Structure Learning in Inverse Ising Problems Using ${\ell }_2$-Regularized Linear Estimator
• Ising Model Selection Using ${\ell }_1$-Regularized Linear Regression: A Statistical Mechanics Analysis