Large Pseudo-Counts and $L_2$-Norm Penalties Are Necessary for the Mean-Field Inference of Ising and Potts Models

Abstract

Mean field (MF) approximation offers a simple, fast way to infer direct interactions between elements in a network of correlated variables, a common, computationally challenging problem with practical applications in fields ranging from physics and biology to the social sciences. However, MF methods achieve their best performance with strong regularization, well beyond Bayesian expectations, an empirical fact that is poorly understood. In this work, we study the influence of pseudo-count and $L_2$-norm regularization schemes on the quality of inferred Ising or Potts interaction networks from correlation data within the MF approximation. We argue, based on the analysis of small systems, that the optimal value of the regularization strength remains finite even if the sampling noise tends to zero, in order to correct for systematic biases introduced by the MF approximation. Our claim is corroborated by extensive numerical studies of diverse model systems and by the analytical study of the $m$-component spin model, for large but finite $m$. Additionally we find that pseudo-count regularization is robust against sampling noise, and often outperforms $L_2$-norm regularization, particularly when the underlying network of interactions is strongly heterogeneous. Much better performances are generally obtained for the Ising model than for the Potts model, for which only couplings incoming onto medium-frequency symbols are reliably inferred.

arXiv comment25 pages, 17 figures

References in corpus (14)

• Weak pairwise correlations imply strongly correlated network states in a neural population
• Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
• Identification of direct residue contacts in protein-protein interaction by message passing
• Universal and non-universal properties of cross-correlations in financial time series
• Statistical mechanics for natural flocks of birds
• High-dimensional Ising model selection using ${\ell }_1$-regularized logistic regression
• Market impact and trading profile of large trading orders in stock markets
• Inverse Ising inference using all the data
• Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data
• From principal component to direct coupling analysis of coevolution in proteins: Low-eigenvalue modes are needed for structure prediction
• Small-correlation expansions for the inverse Ising problem
• High-Dimensional Inference with the generalized Hopfield Model: Principal Component Analysis and Corrections
• Bethe-Peierls approximation and the inverse Ising model
• Adaptive Thouless-Anderson-Palmer approach to inverse Ising problems with quenched random fields

Cited by in corpus (12)

• Coevolutionary landscape inference and the context-dependence of mutations in beta-lactamase TEM-1
• Inverse statistical problems: from the inverse Ising problem to data science
• Influence of Multiple Sequence Alignment Depth on Potts Statistical Models of Protein Covariation
• Sparse generative modeling via parameter-reduction of Boltzmann machines: application to protein-sequence families
• On the entropy of protein families
• Inference of compressed Potts graphical models
• Direct Coupling Analysis of Epistasis in Allosteric Materials
• Effects of hidden nodes on network structure inference
• Optimal regularizations for data generation with probabilistic graphical models
• Breakdown of random matrix universality in Markov models
• Resummed mean-field inference for strongly coupled data
• A Density Consistency approach to the inverse Ising problem