Inverse statistical problems: from the inverse Ising problem to data science

Abstract

Inverse problems in statistical physics are motivated by the challenges of `big data' in different fields, in particular high-throughput experiments in biology. In inverse problems, the usual procedure of statistical physics needs to be reversed: Instead of calculating observables on the basis of model parameters, we seek to infer parameters of a model based on observations. In this review, we focus on the inverse Ising problem and closely related problems, namely how to infer the coupling strengths between spins given observed spin correlations, magnetisations, or other data. We review applications of the inverse Ising problem, including the reconstruction of neural connections, protein structure determination, and the inference of gene regulatory networks. For the inverse Ising problem in equilibrium, a number of controlled and uncontrolled approximate solutions have been developed in the statistical mechanics community. A particularly strong method, pseudolikelihood, stems from statistics. We also review the inverse Ising problem in the non-equilibrium case, where the model parameters must be reconstructed based on non-equilibrium statistics.

Review article, 45 pages

References in corpus (71)

• A nonequilibrium equality for free energy differences
• Detecting topological order in a ground state wave function
• Weak pairwise correlations imply strongly correlated network states in a neural population
• Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
• Machine learning phases of matter
• Measuring entanglement entropy through the interference of quantum many-body twins
• Identification of direct residue contacts in protein-protein interaction by message passing
• Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model
• Statistical mechanics for natural flocks of birds
• Are biological systems poised at criticality?
• Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models
• High-dimensional Ising model selection using ${\ell }_1$-regularized logistic regression
• Statistical physics of inference: Thresholds and algorithms
• Does the 1/f frequency-scaling of brain signals reflect self-organized critical states?
• Maximum entropy models for antibody diversity
• Searching for collective behavior in a network of real neurons
• Information based clustering
• The Ising Model for Neural Data: Model Quality and Approximate Methods for Extracting Functional Connectivity
• Social interactions dominate speed control in driving natural flocks toward criticality
• Perturbation Biology: inferring signaling networks in cellular systems
• Pairwise maximum entropy models for studying large biological systems: when they can and when they can't work
• Loop series for discrete statistical models on graphs
• Inverse Ising inference using all the data
• Prediction of spatio-temporal patterns of neural activity from pairwise correlations
• Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data
• Statistical physics of pairwise probability models
• Small-correlation expansions for the inverse Ising problem
• Mean Field Theory For Non-Equilibrium Network Reconstruction
• Stimulus-dependent maximum entropy models of neural population codes
• Zipf's law and criticality in multivariate data without fine-tuning
• Designed Interaction Potentials via Inverse Methods for Self-Assembly
• The simplest maximum entropy model for collective behavior in a neural network
• Dynamical criticality in the collective activity of a population of retinal neurons
• Adaptive cluster expansion for the inverse Ising problem: convergence, algorithm and tests
• The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods
• On sampling and modeling complex systems
• Dynamical maximum entropy approach to flocking
• Thermodynamics of natural images
• On the criticality of inferred models
• The value of monitoring to control evolving populations
• Pseudolikelihood Decimation Algorithm Improving the Inference of the Interaction Network in a General Class of Ising Models
• The Effect of Nonstationarity on Models Inferred from Neural Data
• Mean-field theory for the inverse Ising problem at low temperatures
• On the sufficiency of pairwise interactions in maximum entropy models of biological networks
• Dynamical TAP equations for non-equilibrium Ising spin glasses
• Growth against entropy in bacterial metabolism: the phenotypic trade-off behind empirical growth rate distributions in E. coli
• Inverse Statistical Mechanics: Probing the Limitations of Isotropic Pair Potentials to Produce Ground-State Structural Extremes
• Maximum likelihood reconstruction for Ising models with asynchronous updates
• Learning and inference in a nonequilibrium Ising model with hidden nodes
• Loop expansion around the Bethe-Peierls approximation for lattice models
• Market structure explained by pairwise interactions
• Network inference using asynchronously updated kinetic Ising Model
• Bethe-Peierls approximation and the inverse Ising model
• Beyond position weight matrices: nucleotide correlations in transcription factor binding sites and their description
• Density-matrix based determination of low-energy model Hamiltonians from ab initio wavefunctions
• Large Pseudo-Counts and $L_2$-Norm Penalties Are Necessary for the Mean-Field Inference of Ising and Potts Models
• Flat histogram Monte Carlo method
• Parameters estimation for spatio-temporal maximum entropy distributions: application to neural spike trains
• Consistent inference of a general model using the pseudo-likelihood method
• U.S. stock market interaction network as learned by the Boltzmann Machine
• Learning Maximum Entropy Models from finite size datasets: a fast Data-Driven algorithm allows sampling from the posterior distribution
• Intrinsic limitations of inverse inference in the pairwise Ising spin glass
• Learning with hidden variables
• Belief-Propagation and replicas for inference and learning in a kinetic Ising model with hidden spins
• A general approximation for the dynamics of quantitative traits
• Inference and learning in sparse systems with multiple states
• Dynamics and Performance of Susceptibility Propagation on Synthetic Data
• A statistical physics perspective on criticality in financial markets
• Sparse model selection in the highly under-sampled regime
• Learning of couplings for random asymmetric kinetic Ising models revisited: random correlation matrices and learning curves
• Resummed mean-field inference for strongly coupled data

Cited by in corpus (106)

• Machine learning and the physical sciences
• A high-bias, low-variance introduction to Machine Learning for physicists
• Inverse Statistical Physics of Protein Sequences: A Key Issues Review
• Solving Statistical Mechanics Using Variational Autoregressive Networks
• Network reconstruction and community detection from dynamics
• Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
• Machine Learning for Condensed Matter Physics
• Global analysis of more than 50,000 SARS-Cov-2 genomes reveals epistasis between 8 viral genes
• Quantitative Immunology for Physicists
• Belief propagation for networks with loops
• Quantum Machine-Learning for Eigenstate Filtration in Two-Dimensional Materials
• Selection of sequence motifs and generative Hopfield-Potts models for protein familiesilies
• Does the brain behave like a (complex) network? I. Dynamics
• Learning the Ising Model with Generative Neural Networks
• Phylogenetic correlations can suffice to infer protein partners from sequences
• Mean-field inference methods for neural networks
• Programmable Quantum Annealers as Noisy Gibbs Samplers
• From real materials to model Hamiltonians with density matrix downfolding
• Mapping of attention mechanisms to a generalized Potts model
• Sparse generative modeling via parameter-reduction of Boltzmann machines: application to protein-sequence families
• Tackling the subsampling problem to infer collective properties from limited data
• Building general Langevin models from discrete data sets
• Assessing the accuracy of direct-coupling analysis for RNA contact prediction
• Inference of compressed Potts graphical models
• Separating intrinsic interactions from extrinsic correlations in a network of sensory neurons
• Introduction to correlation networks: Interdisciplinary approaches beyond thresholding
• Dynamical phase transitions in the nonreciprocal Ising model
• Quantifying Relevance in Learning and Inference
• Memory-free dynamics for the TAP equations of Ising models with arbitrary rotation invariant ensembles of random coupling matrices
• Direct Coupling Analysis of Epistasis in Allosteric Materials
• Inverse Ising problem in continuous time: A latent variable approach
• Higher-order interactions in statistical physics and machine learning: A model-independent solution to the inverse problem at equilibrium
• Aligning biological sequences by exploiting residue conservation and coevolution
• Correlations from structure and phylogeny combine constructively in the inference of protein partners from sequences
• The Role of Data in Model Building and Prediction: A Survey Through Examples
• Energy landscape analysis based on the Ising model: Tutorial review
• Lattice physics approaches for neural networks
• Detecting many-body Bell non-locality by solving Ising models
• Ensemble Inhibition and Excitation in the Human Cortex: an Ising Model Analysis with Uncertainties
• Correlation-Compressed Direct Coupling Analysis
• Causality inference in stochastic systems from neurons to currencies: Profiting from small sample size
• Effects of hidden nodes on the reconstruction of bidirectional networks
• Neural activity in quarks language: Lattice Field Theory for a network of real neurons
• Inferring effective couplings with Restricted Boltzmann Machines
• Inferring entropy from structure
• Inferring genetic fitness from genomic data
• Optimal regularizations for data generation with probabilistic graphical models
• Statistical physics of interacting proteins: impact of dataset size and quality assessed in synthetic sequences
• Synergistic signatures of group mechanisms in higher-order systems
• The Stochastic complexity of spin models: Are pairwise models really simple?
• Network reconstruction via the minimum description length principle
• Inferring Cultural Landscapes with the Inverse Ising Model
• Inferring a network from dynamical signals at its nodes
• Expansion of the effective action around non-Gaussian theories
• The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems
• Inferring couplings in networks across order-disorder phase transitions
• On Maximum Entropy and Inference
• Network science Ising states of matter
• Learning phase transitions from regression uncertainty: A new regression-based machine learning approach for automated detection of phases of matter
• Optimal entanglement witnesses: a scalable data-driven approach
• Enhancing the efficiency of quantum annealing via reinforcement: A path-integral Monte Carlo simulation of the quantum reinforcement algorithm
• Statistical Genetics in and out of Quasi-Linkage Equilibrium (Extended)
• Inferring epistasis from genomic data with comparable mutation and outcrossing rate
• Privacy-preserving machine learning with tensor networks
• From statistical inference to a differential learning rule for stochastic neural networks
• Improved Pseudolikelihood Regularization and Decimation methods on Non-linearly Interacting Systems with Continuous Variables
• Biases in Inverse Ising Estimates of Near-Critical Behaviour
• Learning performance in inverse Ising problems with sparse teacher couplings
• Maximum Entropy Approach for the Prediction of Urban Mobility Patterns
• Watch and learn -- a generalized approach for transferrable learning in deep neural networks via physical principles
• Exact solution to the random sequential dynamics of a message passing algorithm
• Inference of stochastic time series with missing data
• Thermodynamics of the Ising model encoded in restricted Boltzmann machines
• Higher-Order Interactions and Their Duals Reveal Synergy and Logical Dependence beyond Shannon-Information
• A Density Consistency approach to the inverse Ising problem
• Inferring entropy production in many-body systems using nonequilibrium maximum entropy
• Understanding Protein-Complex Assembly through Grand Canonical Maximum Entropy Modeling
• Inverse Ising inference from high-temperature re-weighting of observations
• Correspondence between temporal correlations in time series, inverse problems, and the Spherical Model
• The statistical mechanics of Twitter
• The inverse problem beyond two-body interaction: the cubic mean-field Ising model
• A Very Effective and Simple Diffusion Reconstruction for the Diluted Ising Model
• The Bayesian Inversion Problem for Thermal Average Sampling of Quantum Systems
• Limits on Inferring the Past
• Ancestral Sequence Reconstruction for Co-evolutionary models
• Tailoring the overlap distribution in driven mean-field spin models
• Unsupervised Learning of Effective Quantum Impurity Models
• Inferring Higher-Order Couplings with Neural Networks
• Active image restoration
• Statistical Physics of Medical Diagnostics: Study of a Probabilistic Model
• Inferring the parameters of a Markov process from snapshots of the steady state
• The cavity method to protein design problem
• A perspective on Lindblad's Non-Equilibrium Entropy
• Structure Learning in Inverse Ising Problems Using ${\ell }_2$-Regularized Linear Estimator
• Reveal flocking phase transition of self-propelled active particles by machine learning regression uncertainty
• Nonequilibrium thermodynamics of self-supervised learning
• Explaining the effects of non-convergent sampling in the training of Energy-Based Models
• Mapping Inter-City Trade Networks to Maximum Entropy Models using Electronic Invoice Data
• Inverse Ising inference, hyperuniformity and absorbing states in the Manna model
• Monte Carlo, blocking, and inference: How to measure the renormalization group flow
• Renewal model for dependent binary sequences
• Functional Renormalization for Signal Detection: Dimensional Analysis and Dimensional Phase Transition for Nearly Continuous Spectra Effective Field Theory
• Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings
• Bounded-Error Quantum Simulation via Hamiltonian and Lindbladian Learning
• Classification of SARS-CoV-2 Variants through The Epistatical Circos Plots with Convolutional Neural Networks
• Ising Model Selection Using ${\ell }_1$-Regularized Linear Regression: A Statistical Mechanics Analysis