High-Dimensional Inference with the generalized Hopfield Model: Principal Component Analysis and Corrections

Abstract

We consider the problem of inferring the interactions between a set of N binary variables from the knowledge of their frequencies and pairwise correlations. The inference framework is based on the Hopfield model, a special case of the Ising model where the interaction matrix is defined through a set of patterns in the variable space, and is of rank much smaller than N. We show that Maximum Lik elihood inference is deeply related to Principal Component Analysis when the amp litude of the pattern components, xi, is negligible compared to N^1/2. Using techniques from statistical mechanics, we calculate the corrections to the patterns to the first order in xi/N^1/2. We stress that it is important to generalize the Hopfield model and include both attractive and repulsive patterns, to correctly infer networks with sparse and strong interactions. We present a simple geometrical criterion to decide how many attractive and repulsive patterns should be considered as a function of the sampling noise. We moreover discuss how many sampled configurations are required for a good inference, as a function of the system size, N and of the amplitude, xi. The inference approach is illustrated on synthetic and biological data.

Physical Review E: Statistical, Nonlinear, and Soft Matter Physics (2011) to appear

References in corpus (5)

• Identification of direct residue contacts in protein-protein interaction by message passing
• High-dimensional Ising model selection using ${\ell }_1$-regularized logistic regression
• The Ising Model for Neural Data: Model Quality and Approximate Methods for Extracting Functional Connectivity
• Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data
• Message passing algorithms for the Hopfield network reconstruction: threshold behavior and limitation

Cited by in corpus (22)

• Statistical mechanics for natural flocks of birds
• Inverse Statistical Physics of Protein Sequences: A Key Issues Review
• From principal component to direct coupling analysis of coevolution in proteins: Low-eigenvalue modes are needed for structure prediction
• Phase transitions in Restricted Boltzmann Machines with generic priors
• On the sufficiency of pairwise interactions in maximum entropy models of biological networks
• Supervised Hebbian Learning
• Beyond position weight matrices: nucleotide correlations in transcription factor binding sites and their description
• Revealing evolutionary constraints on proteins through sequence analysis
• Annealing and replica-symmetry in Deep Boltzmann Machines
• Large Pseudo-Counts and $L_2$-Norm Penalties Are Necessary for the Mean-Field Inference of Ising and Potts Models
• Statistical Physics and Representations in Real and Artificial Neural Networks
• Infinite family of second-law-like inequalities
• Correlated fluctuations in strongly-coupled binary networks beyond equilibrium
• Inverse spin glass and related maximum entropy problems
• Improving landscape inference by integrating heterogeneous data in the inverse Ising problem
• Unsupervised feature learning from finite data by message passing: discontinuous versus continuous phase transition
• Minimal model of permutation symmetry in unsupervised learning
• Estimating the principal components of correlation matrices from all their empirical eigenvectors
• Hopfield model with planted patterns: a teacher-student self-supervised learning model
• Resummed mean-field inference for strongly coupled data
• Sparse Hopfield network reconstruction with ${\ell }_1$ regularization
• Impact of phylogeny on the inference of functional sectors from protein sequence data