6 citations · 14 across the 20 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
A unified convergence theory for adaptive first-order methods in the nonconvex case, including AdaNorm, full and diagonal AdaGrad and Muon
S. Gratton, Ph. L. Toint
A unified framework for first-order optimization algorithms fornonconvex unconstrained optimization is proposed that uses adaptivelypreconditioned gradients and includes popular me…
cs.LG2023
Divergence of the ADAM algorithm with fixed-stepsize: a (very) simple example
Ph. L. Toint
A very simple unidimensional function with Lipschitz continuous gradient is constructed such that the ADAM algorithm with constant stepsize, started from the origin, diverges when…