3 papers
cs.LG2026
Variational Structure at the Edge of Stability
Eric Regis
When discrete-time optimizers operate at the edge of stability, they exhibit near-two-periodic behavior. These oscillatory dynamics are reminiscent of conservative systems, such as…
cs.LG2026
A Rod Flow Model for Adam at the Edge of Stability
Eric Regis, Sinho Chewi
Cohen et al. (arXiv:2207.14484) observed that adaptive gradient methods such as Adam operate at the edge of stability. While there has been significant work on continuous-time mode…
cs.LG2026
Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stability
Eric Regis, Sinho Chewi
How can we understand gradient-based training over non-convex landscapes? The edge of stability phenomenon, introduced in Cohen et al. (2021), indicates that the answer is not so s…