2 papers
cs.LG2025
Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability
Kaiqi Jiang, Jeremy Cohen, Yuanzhi Li
The study of Neural Tangent Kernels (NTKs) in deep learning has drawn increasing attention in recent years. NTKs typically actively change during training and are related to featur…
cs.LG2021
Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability
Jeremy M. Cohen, Simran Kaur, Yuanzhi Li +2
We empirically demonstrate that full-batch gradient descent on neural network training objectives typically operates in a regime we call the Edge of Stability. In this regime, the…