Gradient descent dynamics and the jamming transition in infinite dimensions

Abstract

Gradient descent dynamics in complex energy landscapes, i.e. featuring multiple minima, finds application in many different problems, from soft matter to machine learning. Here, we analyze one of the simplest examples, namely that of soft repulsive particles in the limit of infinite spatial dimension $d$. The gradient descent dynamics then displays a jamming transition: at low density, it reaches zero-energy states in which particles' overlaps are fully eliminated, while at high density the energy remains finite and overlaps persist. At the transition, the dynamics becomes critical. In the $d\to \infty$ limit, a set of self-consistent dynamical equations can be derived via mean field theory. We analyze these equations and we present some partial progress towards their solution. We also study the Random Lorentz Gas in a range of $d=2\dots 22$, and obtain a robust estimate for the jamming transition in $d\to \infty$. The jamming transition is analogous to the capacity transition in supervised learning, and in the appendix we discuss this analogy in the case of a simple one-layer fully-connected perceptron.

arXiv comment31 pages, 7 figures

References in corpus (63)

• Jamming at Zero Temperature and Zero Applied Stress: the Epitome of Disorder
• Analytical Solution of the Off-Equilibrium Dynamics of a Long Range Spin-Glass Model
• Mean field theory of hard sphere glasses and jamming
• Random Packings of Frictionless Particles
• Fractal free energy landscapes in structural glasses
• On the Out of Equilibrium Relaxation of the Sherrington - Kirkpatrick model
• The structural glass transition and the entropy of the metastable states
• Critical Scaling of Shear Viscosity at the Jamming Transition
• Cleaning large correlation matrices: tools from random matrix theory
• Marginal Stability in Structural, Spin and Electron Glasses
• Universal Spectrum of Normal Modes in Low-Temperature Glasses: an Exact Solution
• On the nature of the low-temperature phase in discontinuous mean-field spin glasses
• Exact theory of dense amorphous hard spheres in high dimension. III. The full RSB solution
• Jamming versus Glass Transitions
• Stability at Random Close Packing
• Jamming Criticality Revealed by Removing Localized Buckling Excitations
• A collective phase in resource competition in a highly diverse ecosystem
• The simplest model of jamming
• Solution of the dynamics of liquids in the large-dimensional limit
• Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems
• Numerical implementation of dynamical mean field theory for disordered systems: application to the Lotka-Volterra model of ecosystems
• On the cooling-schedule dependence of the dynamics of mean-field glasses
• Temperature evolution and bifurcations of metastable states in mean-field spin glasses, with connections with structural glasses
• Microscopic theory of the jamming transition of harmonic spheres
• Dynamical transition of glasses: from exact to approximate
• A jamming transition from under- to over-parametrization affects loss landscape and generalization
• May's Instability in Large Economies
• Exploring the jamming transition over a wide range of critical densities
• Rethinking mean-field glassy dynamics and its relation with the energy landscape: the awkward case of the spherical mixed p-spin model
• Replica-symmetry-breaking transitions and off-equilibrium dynamics
• Out-of-equilibrium dynamical mean-field equations for the perceptron model
• Jamming in multilayer supervised learning models
• Comparing Dynamics: Deep Neural Networks versus Glassy Systems
• A simple theory for the dynamics of mean-field-like models of glass-forming fluids
• Low-frequency vibrations of jammed packings in large spatial dimensions
• Marginally stable phases in mean-field structural glasses
• Relaxation dynamics in the energy landscape of glass-forming liquids
• Quantitative approximation schemes for glasses
• Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem
• Numerical solution of the dynamical mean field theory of infinite-dimensional equilibrium liquids
• Strong ergodicity breaking in aging of mean field spin glasses
• Constraint satisfaction mechanisms for marginal stability and criticality in large ecosystems
• Out-of-equilibrium dynamical equations of infinite-dimensional particle systems. I. The isotropic case
• Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference
• Marginal stability of soft anharmonic mean field spin glasses
• How to iron out rough landscapes and get optimal performances: Averaged Gradient Descent and its application to tensor PCA
• Universal relaxation dynamics of sphere packings below jamming
• Interplay between percolation and glassiness in the random Lorentz gas
• On the Dynamics of Liquids in the Large-Dimensional Limit
• Mean-field caging in a random Lorentz gas
• Dynamical Mean-Field Theory and Aging Dynamics
• Slow coarsening in jammed athermal soft particle suspensions
• Relaxation dynamics of non-Brownian spheres below jamming
• The dimensional evolution of structure and dynamics in hard sphere liquids
• Mean-Field Predictions of Scaling Prefactors Match Low-Dimensional Jammed Packings
• Dynamic length scales in athermal, shear-driven, jamming of frictionless disks in two dimensions
• Introduction to the dynamics of disordered systems: equilibrium and gradient descent
• Systems of random linear equations and the phase transition in MacArthur's resource-competition model
• High dimensional optimization under non-convex excluded volume constraints
• Surfing on minima of isostatic landscapes: avalanches and unjamming transition
• Force balance controls the relaxation time of the gradient descent algorithm in the satisfiable phase
• Good speciation and endogenous business cycles in a constraint satisfaction macroeconomic model
• Note: Relaxation time below jamming