Variational perturbation and extended Plefka approaches to dynamics on random networks: the case of the kinetic Ising model

Abstract

We describe and analyze some novel approaches for studying the dynamics of Ising spin glass models. We first briefly consider the variational approach based on minimizing the Kullback-Leibler divergence between independent trajectories and the real ones and note that this approach only coincides with the mean field equations from the saddle point approximation to the generating functional when the dynamics is defined through a logistic link function, which is the case for the kinetic Ising model with parallel update. We then spend the rest of the paper developing two ways of going beyond the saddle point approximation to the generating functional. In the first one, we develop a variational perturbative approximation to the generating functional by expanding the action around a quadratic function of the local fields and conjugate local fields whose parameters are optimized. We derive analytical expressions for the optimal parameters and show that when the optimization is suitably restricted, we recover the mean field equations that are exact for the fully asymmetric random couplings (Mézard and Sakellariou, 2011). However, without this restriction the results are different. We also describe an extended Plefka expansion in which in addition to the magnetization, we also fix the correlation and response functions. Finally, we numerically study the performance of these approximations for Sherrington-Kirkpatrick type couplings for various coupling strengths, degrees of coupling symmetry and external fields. We show that the dynamical equations derived from the extended Plefka expansion outperform the others in all regimes, although it is computationally more demanding. The unconstrained variational approach does not perform well in the small coupling regime, while it approaches dynamical TAP equations of (Roudi and Hertz, 2011) for strong couplings.

References in corpus (9)

• Mean Field Theory For Non-Equilibrium Network Reconstruction
• Dynamical TAP equations for non-equilibrium Ising spin glasses
• On the formal equivalence of the TAP and thermodynamic methods in the SK model
• Learning and inference in a nonequilibrium Ising model with hidden nodes
• Dynamical TAP approach to mean field glassy systems
• Dynamic mean-field and cavity methods for diluted Ising systems
• Effect of coupling asymmetry on mean-field solutions of direct and inverse Sherrington-Kirkpatrick model
• Generalized mean field approximation for parallel dynamics of the Ising model
• Dynamics of asymmetric kinetic Ising systems revisited

Cited by in corpus (11)

• Path integral methods for the dynamics of stochastic and disordered systems
• A unifying framework for mean-field theories of asymmetric kinetic Ising systems
• Tackling the subsampling problem to infer collective properties from limited data
• A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
• Statistical physics approaches to subnetwork dynamics in biochemical systems
• Inferring hidden states in Langevin dynamics on large networks: Average case performance
• Inference for dynamics of continuous variables: the Extended Plefka Expansion with hidden nodes
• A matrix product algorithm for stochastic dynamics on networks, applied to non-equilibrium Glauber dynamics
• The matrix product approximation for the dynamic cavity method
• Critical scaling in hidden state inference for linear Langevin dynamics
• Inference in neural networks using conditional mean-field methods