Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings

Abstract

Dynamical mean-field theory is a powerful physics tool used to analyze the typical behavior of neural networks, where neurons can be recurrently connected, or multiple layers of neurons can be stacked. However, it is not easy for beginners to access the essence of this tool and the underlying physics. Here, we give a pedagogical introduction of this method in a particular example of random neural networks, where neurons are randomly and fully connected by correlated synapses and therefore the network exhibits rich emergent collective dynamics. We also review related past and recent important works applying this tool. In addition, a physically transparent and alternative method, namely the dynamical cavity method, is also introduced to derive exactly the same results. The numerical implementation of solving the integro-differential mean-field equations is also detailed, with an illustration of exploring the fluctuation dissipation theorem.

arXiv comment27 pages, 5 figures, 44 references, revised version for SciPost Physics Lecture Notes

References in corpus (19)

• Theoretical perspective on the glass transition and amorphous materials
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Analytical Solution of the Off-Equilibrium Dynamics of a Long Range Spin-Glass Model
• Spin-Glass Theory for Pedestrians
• The effective temperature
• Stochastic Dynamical Structure (SDS) of Nonequilibrium Processes in the Absence of Detailed Balance. II: construction of SDS with nonlinear force and multiplicative noise
• Numerical implementation of dynamical mean field theory for disordered systems: application to the Lotka-Volterra model of ecosystems
• Dynamics of random recurrent networks with correlated low-rank structure
• Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks
• Path Integral Approach to Random Neural Networks
• Path integral methods for the dynamics of stochastic and disordered systems
• Out-of-equilibrium dynamical mean-field equations for the perceptron model
• Learning Curves for Deep Neural Networks: A Gaussian Field Theory Perspective
• Fluctuation-dissipation relations for spiking neurons
• Unified field theoretical approach to deep and recurrent neuronal networks
• Extended Plefka Expansion for Stochastic Dynamics
• Spectrum of non-Hermitian deep-Hebbian neural networks
• Dynamics of asymmetric kinetic Ising systems revisited
• Emergence of hierarchical modes from deep learning

Cited by in corpus (9)

• Eight challenges in developing theory of intelligence
• Nonequilbrium physics of generative diffusion models
• An optimization-based equilibrium measure describes non-equilibrium steady state dynamics: application to edge of chaos
• Dynamical Theory for Adaptive Systems
• Neuronal correlations shape the scaling behavior of memory capacity and nonlinear computational capability of reservoir recurrent neural networks
• Network reconstruction may not mean dynamics prediction
• Synaptic plasticity alters the nature of chaos transition in neural networks
• Dynamical mean field approach to associative memory model with non-monotonic transfer functions
• Response function as a quantitative measure of consciousness in brain dynamics