Inverse statistical problems: from the inverse Ising problem to data science

Abstract

Inverse problems in statistical physics are motivated by the challenges of `big data' in different fields, in particular high-throughput experiments in biology. In inverse problems, the usual procedure of statistical physics needs to be reversed: Instead of calculating observables on the basis of model parameters, we seek to infer parameters of a model based on observations. In this review, we focus on the inverse Ising problem and closely related problems, namely how to infer the coupling strengths between spins given observed spin correlations, magnetisations, or other data. We review applications of the inverse Ising problem, including the reconstruction of neural connections, protein structure determination, and the inference of gene regulatory networks. For the inverse Ising problem in equilibrium, a number of controlled and uncontrolled approximate solutions have been developed in the statistical mechanics community. A particularly strong method, pseudolikelihood, stems from statistics. We also review the inverse Ising problem in the non-equilibrium case, where the model parameters must be reconstructed based on non-equilibrium statistics.

Review article, 45 pages

References in corpus (20)

• Identification of direct residue contacts in protein-protein interaction by message passing
• Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model
• Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models
• High-dimensional Ising model selection using ${\ell }_1$-regularized logistic regression
• Does the 1/f frequency-scaling of brain signals reflect self-organized critical states?
• The Ising Model for Neural Data: Model Quality and Approximate Methods for Extracting Functional Connectivity
• Loop series for discrete statistical models on graphs
• Prediction of spatio-temporal patterns of neural activity from pairwise correlations
• Small-correlation expansions for the inverse Ising problem
• Mean Field Theory For Non-Equilibrium Network Reconstruction
• Designed Interaction Potentials via Inverse Methods for Self-Assembly
• Dynamical criticality in the collective activity of a population of retinal neurons
• The value of monitoring to control evolving populations
• Mean-field theory for the inverse Ising problem at low temperatures
• Dynamical TAP equations for non-equilibrium Ising spin glasses
• Inverse Statistical Mechanics: Probing the Limitations of Isotropic Pair Potentials to Produce Ground-State Structural Extremes
• Large Pseudo-Counts and $L_2$-Norm Penalties Are Necessary for the Mean-Field Inference of Ising and Potts Models
• Parameters estimation for spatio-temporal maximum entropy distributions: application to neural spike trains
• Belief-Propagation and replicas for inference and learning in a kinetic Ising model with hidden spins
• Dynamics and Performance of Susceptibility Propagation on Synthetic Data

Cited by in corpus (106)

• Machine learning and the physical sciences
• A high-bias, low-variance introduction to Machine Learning for physicists
• Inverse Statistical Physics of Protein Sequences: A Key Issues Review
• Solving Statistical Mechanics Using Variational Autoregressive Networks
• Network reconstruction and community detection from dynamics
• Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
• Machine Learning for Condensed Matter Physics
• Global analysis of more than 50,000 SARS-Cov-2 genomes reveals epistasis between 8 viral genes
• Quantitative Immunology for Physicists
• Belief propagation for networks with loops
• Quantum Machine-Learning for Eigenstate Filtration in Two-Dimensional Materials
• Selection of sequence motifs and generative Hopfield-Potts models for protein familiesilies
• Does the brain behave like a (complex) network? I. Dynamics
• Learning the Ising Model with Generative Neural Networks
• Phylogenetic correlations can suffice to infer protein partners from sequences
• Mean-field inference methods for neural networks
• Programmable Quantum Annealers as Noisy Gibbs Samplers
• From real materials to model Hamiltonians with density matrix downfolding
• Mapping of attention mechanisms to a generalized Potts model
• Sparse generative modeling via parameter-reduction of Boltzmann machines: application to protein-sequence families
• Tackling the subsampling problem to infer collective properties from limited data
• Building general Langevin models from discrete data sets
• Assessing the accuracy of direct-coupling analysis for RNA contact prediction
• Inference of compressed Potts graphical models
• Separating intrinsic interactions from extrinsic correlations in a network of sensory neurons
• Introduction to correlation networks: Interdisciplinary approaches beyond thresholding
• Dynamical phase transitions in the nonreciprocal Ising model
• Quantifying Relevance in Learning and Inference
• Memory-free dynamics for the TAP equations of Ising models with arbitrary rotation invariant ensembles of random coupling matrices
• Direct Coupling Analysis of Epistasis in Allosteric Materials
• Inverse Ising problem in continuous time: A latent variable approach
• Higher-order interactions in statistical physics and machine learning: A model-independent solution to the inverse problem at equilibrium
• Aligning biological sequences by exploiting residue conservation and coevolution
• Correlations from structure and phylogeny combine constructively in the inference of protein partners from sequences
• The Role of Data in Model Building and Prediction: A Survey Through Examples
• Energy landscape analysis based on the Ising model: Tutorial review
• Lattice physics approaches for neural networks
• Detecting many-body Bell non-locality by solving Ising models
• Ensemble Inhibition and Excitation in the Human Cortex: an Ising Model Analysis with Uncertainties
• Correlation-Compressed Direct Coupling Analysis
• Causality inference in stochastic systems from neurons to currencies: Profiting from small sample size
• Effects of hidden nodes on the reconstruction of bidirectional networks
• Neural activity in quarks language: Lattice Field Theory for a network of real neurons
• Inferring effective couplings with Restricted Boltzmann Machines
• Inferring entropy from structure
• Inferring genetic fitness from genomic data
• Optimal regularizations for data generation with probabilistic graphical models
• Statistical physics of interacting proteins: impact of dataset size and quality assessed in synthetic sequences
• Synergistic signatures of group mechanisms in higher-order systems
• The Stochastic complexity of spin models: Are pairwise models really simple?
• Network reconstruction via the minimum description length principle
• Inferring Cultural Landscapes with the Inverse Ising Model
• Inferring a network from dynamical signals at its nodes
• Expansion of the effective action around non-Gaussian theories
• The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems
• Inferring couplings in networks across order-disorder phase transitions
• On Maximum Entropy and Inference
• Network science Ising states of matter
• Learning phase transitions from regression uncertainty: A new regression-based machine learning approach for automated detection of phases of matter
• Optimal entanglement witnesses: a scalable data-driven approach
• Enhancing the efficiency of quantum annealing via reinforcement: A path-integral Monte Carlo simulation of the quantum reinforcement algorithm
• Statistical Genetics in and out of Quasi-Linkage Equilibrium (Extended)
• Inferring epistasis from genomic data with comparable mutation and outcrossing rate
• Privacy-preserving machine learning with tensor networks
• From statistical inference to a differential learning rule for stochastic neural networks
• Improved Pseudolikelihood Regularization and Decimation methods on Non-linearly Interacting Systems with Continuous Variables
• Biases in Inverse Ising Estimates of Near-Critical Behaviour
• Learning performance in inverse Ising problems with sparse teacher couplings
• Maximum Entropy Approach for the Prediction of Urban Mobility Patterns
• Watch and learn -- a generalized approach for transferrable learning in deep neural networks via physical principles
• Exact solution to the random sequential dynamics of a message passing algorithm
• Inference of stochastic time series with missing data
• Thermodynamics of the Ising model encoded in restricted Boltzmann machines
• Higher-Order Interactions and Their Duals Reveal Synergy and Logical Dependence beyond Shannon-Information
• A Density Consistency approach to the inverse Ising problem
• Inferring entropy production in many-body systems using nonequilibrium maximum entropy
• Understanding Protein-Complex Assembly through Grand Canonical Maximum Entropy Modeling
• Inverse Ising inference from high-temperature re-weighting of observations
• Correspondence between temporal correlations in time series, inverse problems, and the Spherical Model
• The statistical mechanics of Twitter
• The inverse problem beyond two-body interaction: the cubic mean-field Ising model
• A Very Effective and Simple Diffusion Reconstruction for the Diluted Ising Model
• The Bayesian Inversion Problem for Thermal Average Sampling of Quantum Systems
• Limits on Inferring the Past
• Ancestral Sequence Reconstruction for Co-evolutionary models
• Tailoring the overlap distribution in driven mean-field spin models
• Unsupervised Learning of Effective Quantum Impurity Models
• Inferring Higher-Order Couplings with Neural Networks
• Active image restoration
• Statistical Physics of Medical Diagnostics: Study of a Probabilistic Model
• Inferring the parameters of a Markov process from snapshots of the steady state
• The cavity method to protein design problem
• A perspective on Lindblad's Non-Equilibrium Entropy
• Structure Learning in Inverse Ising Problems Using ${\ell }_2$-Regularized Linear Estimator
• Reveal flocking phase transition of self-propelled active particles by machine learning regression uncertainty
• Nonequilibrium thermodynamics of self-supervised learning
• Explaining the effects of non-convergent sampling in the training of Energy-Based Models
• Mapping Inter-City Trade Networks to Maximum Entropy Models using Electronic Invoice Data
• Inverse Ising inference, hyperuniformity and absorbing states in the Manna model
• Monte Carlo, blocking, and inference: How to measure the renormalization group flow
• Renewal model for dependent binary sequences
• Functional Renormalization for Signal Detection: Dimensional Analysis and Dimensional Phase Transition for Nearly Continuous Spectra Effective Field Theory
• Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings
• Bounded-Error Quantum Simulation via Hamiltonian and Lindbladian Learning
• Classification of SARS-CoV-2 Variants through The Epistatical Circos Plots with Convolutional Neural Networks
• Ising Model Selection Using ${\ell }_1$-Regularized Linear Regression: A Statistical Mechanics Analysis