Out-of-equilibrium dynamical mean-field equations for the perceptron model
arXiv:1710.04894 · doi:10.1088/1751-8121/aaa68d
Abstract
Perceptrons are the building blocks of many theoretical approaches to a wide range of complex systems, ranging from neural networks and deep learning machines, to constraint satisfaction problems, glasses and ecosystems. Despite their applicability and importance, a detailed study of their Langevin dynamics has never been performed yet. Here we derive the mean-field dynamical equations that describe the continuous random perceptron in the thermodynamic limit, in a very general setting with arbitrary noise and friction kernels, not necessarily related by equilibrium relations. We derive the equations in two ways: via a dynamical cavity method, and via a path-integral approach in its supersymmetric formulation. The end point of both approaches is the reduction of the dynamics of the system to an effective stochastic process for a representative dynamical variable. Because the perceptron is formally very close to a system of interacting particles in a high dimensional space, the methods we develop here can be transferred to the study of liquid and glasses in high dimensions. Potentially interesting applications are thus the study of the glass transition in active matter, the study of the dynamics around the jamming transition, and the calculation of rheological properties in driven systems.
References in corpus (41)
- Theoretical perspective on the glass transition and amorphous materials
- Analytical Solution of the Off-Equilibrium Dynamics of a Long Range Spin-Glass Model
- The Bethe lattice spin glass revisited
- Supercooled Liquids for Pedestrians
- Gibbs States and the Set of Solutions of Random Constraint Satisfaction Problems
- Fractal free energy landscapes in structural glasses
- On the Out of Equilibrium Relaxation of the Sherrington - Kirkpatrick model
- Spin-Glass Theory for Pedestrians
- Statistical physics of inference: Thresholds and algorithms
- Critical Scaling of Shear Viscosity at the Jamming Transition
- Theory of nonlinear rheology and yielding of dense colloidal suspensions
- Nonequilibrium glass transitions in driven and active matter
- Mode-Coupling Theory (MCT) Lecture Notes
- Glass and Jamming Transitions: From Exact Results to Finite-Dimensional Descriptions
- Two-time scales, two-temperature scenario for nonlinear rheology
- Universal Spectrum of Normal Modes in Low-Temperature Glasses: an Exact Solution
- Nonequilibrium glassy dynamics of self-propelled hard disks
- Glass Rheology: From mode-coupling theory to a dynamical yield criterion
- A collective phase in resource competition in a highly diverse ecosystem
- The simplest model of jamming
- Solution of the dynamics of liquids in the large-dimensional limit
- Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems
- The nonequilibrium glassy dynamics of self-propelled particles
- Disentangling glass and jamming physics in the rheology of soft materials
- Fluctuation theorem for non-equilibrium relaxational systems driven by external forces
- Unified Theory of Inertial Granular Flows and Non-Brownian Suspensions
- Dynamics within metastable states in a mean-field spin glass
- A theory for the dynamics of dense systems of athermal self-propelled particles
- Generalization of the cavity method for adiabatic evolution of Gibbs states
- Avalanche dynamics of elastic interfaces
- Diverging viscosity and soft granular rheology in non-Brownian suspensions
- Mean-field avalanches in jammed spheres
- Non equilibrium mode coupling theory for supercooled liquids and glasses
- A simple theory for the dynamics of mean-field-like models of glass-forming fluids
- Statics and dynamics of infinite-dimensional liquids and glasses: a parallel, compact derivation
- The jamming transition in high dimension: an analytical study of the TAP equations and the effective thermodynamic potential
- Diffusion and Velocity Auto-Correlation in Shearing Granular Media
- Techniques of replica symmetry breaking and the storage problem of the McCulloch-Pitts neuron
- Non equilibrium dynamics of isolated disordered systems: the classical Hamiltonian p-spin model
- Yield stress in amorphous solids: A mode-coupling theory analysis
- Shearing a glass and the role of pin-delay in models of interface depinning
Cited by in corpus (33)
- Dimension of activity in random neural networks
- The effective noise of Stochastic Gradient Descent
- Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem
- Numerical solution of the dynamical mean field theory of infinite-dimensional equilibrium liquids
- Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference
- Out-of-equilibrium dynamical equations of infinite-dimensional particle systems. I. The isotropic case
- Memory formation in jammed hard spheres
- On the Dynamics of Liquids in the Large-Dimensional Limit
- On weak ergodicity breaking in mean-field spin glasses
- Mean-field caging in a random Lorentz gas
- Dynamical Mean-Field Theory and Aging Dynamics
- Aging by near-extinctions in many-variable interacting populations
- Out-of-equilibrium dynamical equations of infinite-dimensional particle systems. II. The anisotropic case under shear strain
- Impact of jamming criticality on low-temperature anomalies in structural glasses
- Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings
- A continuous constraint satisfaction problem for the rigidity transition in confluent tissues
- Rigorous dynamical mean field theory for stochastic gradient descent methods
- Introduction to the dynamics of disordered systems: equilibrium and gradient descent
- Surfing on minima of isostatic landscapes: avalanches and unjamming transition
- Phase transitions in the mini-batch size for sparse and dense two-layer neural networks
- Force balance controls the relaxation time of the gradient descent algorithm in the satisfiable phase
- Jamming, relaxation, and memory in a structureless glass former
- Dynamical mean field theory for models of confluent tissues and beyond
- Tractability from overparametrization: The example of the negative perceptron
- Thresholds of descending algorithms in inference problems
- Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification
- High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models
- Dynamics and fluctuations of minimally-structured glass formers
- Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems
- Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model
- Comment on "Explicit Analytical Solution for Random Close Packing in and "
- Simplified derivations for high-dimensional convex learning problems
- Gradient descent dynamics and the jamming transition in infinite dimensions