Statistical field theory for neural networks

Abstract

These notes attempt a self-contained introduction into statistical field theory applied to neural networks of rate units and binary spins. The presentation consists of three parts: First, the introduction of fundamental notions of probabilities, moments, cumulants, and their relation by the linked cluster theorem, of which Wick's theorem is the most important special case; followed by the diagrammatic formulation of perturbation theory, reviewed in the statistical setting. Second, dynamics described by stochastic differential equations in the Ito-formulation, treated in the Martin-Siggia-Rose-De Dominicis-Janssen path integral formalism. With concepts from disordered systems, we then study networks with random connectivity and derive their self-consistent dynamic mean-field theory, explaining the statistics of fluctuations and the emergence of different phases with regular and chaotic dynamics. Third, we introduce the effective action, vertex functions, and the loopwise expansion. These tools are illustrated by systematic derivations of self-consistency equations, going beyond the mean-field approximation. These methods are applied to the pairwise maximum entropy (Ising spin) model, including the recently-found diagrammatic derivation of the Thouless-Anderson-Palmer mean field theory.

Cited by in corpus (12)

• Nonperturbative renormalization for the neural network-QFT correspondence
• Transient chaotic dimensionality expansion by recurrent networks
• Gell-Mann-Low criticality in neural networks
• Large Deviations Approach to Random Recurrent Neuronal Networks: Parameter Inference and Fluctuation-Induced Transitions
• Extended Anderson Criticality in Heavy-Tailed Neural Networks
• A Microscopic Theory of Intrinsic Timescales in Spiking Neural Networks
• Cumulant mapping as the basis of multi-dimensional spectrometry
• Decomposing neural networks as mappings of correlation functions
• The Two-Particle Irreducible Effective Action for Classical Stochastic Processes
• Theory of the asynchronous state of structured rotator networks and its application to recurrent networks of excitatory and inhibitory units
• Statistical Field Theory and Networks of Spiking Neurons
• On quantum neural networks