Gradient descent dynamics and the jamming transition in infinite dimensions

Abstract

Gradient descent dynamics in complex energy landscapes, i.e. featuring multiple minima, finds application in many different problems, from soft matter to machine learning. Here, we analyze one of the simplest examples, namely that of soft repulsive particles in the limit of infinite spatial dimension $d$. The gradient descent dynamics then displays a jamming transition: at low density, it reaches zero-energy states in which particles' overlaps are fully eliminated, while at high density the energy remains finite and overlaps persist. At the transition, the dynamics becomes critical. In the $d\to \infty$ limit, a set of self-consistent dynamical equations can be derived via mean field theory. We analyze these equations and we present some partial progress towards their solution. We also study the Random Lorentz Gas in a range of $d=2\dots 22$, and obtain a robust estimate for the jamming transition in $d\to \infty$. The jamming transition is analogous to the capacity transition in supervised learning, and in the appendix we discuss this analogy in the case of a simple one-layer fully-connected perceptron.

31 pages, 7 figures

References in corpus (18)

• Jamming at Zero Temperature and Zero Applied Stress: the Epitome of Disorder
• Fractal free energy landscapes in structural glasses
• Cleaning large correlation matrices: tools from random matrix theory
• Marginal Stability in Structural, Spin and Electron Glasses
• Jamming versus Glass Transitions
• Jamming Criticality Revealed by Removing Localized Buckling Excitations
• A collective phase in resource competition in a highly diverse ecosystem
• Relaxation dynamics in the energy landscape of glass-forming liquids
• Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem
• Marginal stability of soft anharmonic mean field spin glasses
• On the Dynamics of Liquids in the Large-Dimensional Limit
• Mean-field caging in a random Lorentz gas
• Dynamical Mean-Field Theory and Aging Dynamics
• The dimensional evolution of structure and dynamics in hard sphere liquids
• Introduction to the dynamics of disordered systems: equilibrium and gradient descent
• High dimensional optimization under non-convex excluded volume constraints
• Surfing on minima of isostatic landscapes: avalanches and unjamming transition
• Note: Relaxation time below jamming