Showing 2026Show all
3 papers · 1 filter
cs.LG2026
Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima
Lachlan Ewen MacDonald, René Vidal
An important quantity in the theory of gradient descent (GD) is the \emph{sharpness}, defined as the largest eigenvalue of the objective Hessian. Classical analyses typically requi…
stat.ML2026
Convergence of Large-Stepsize SGD for the Cross-entropy Loss via Stochastic Lyapunov Stability
Konstantinos Emmanouilidis, Lachlan MacDonald, Salma Tarmoun +1
Modern deep learning has been shown to operate using learning rates far larger than those justified by classical optimization theory. Most prior analyses of this phenomenon focus o…
cs.LG2026
Transformers Learn the Optimal DDPM Denoiser for Multi-Token GMMs
Hongkang Li, Hancheng Min, Rene Vidal
Transformer-based diffusion models have demonstrated remarkable performance at generating high-quality samples. However, our theoretical understanding of the reasons for this succe…