2 papers
math.OC2024
Convergence of the Iterates for Momentum and RMSProp for Local Smooth Functions: Adaptation is the Key
Bilel Bensaid, Gaël Poëtte, Rodolphe Turpault
Both accelerated and adaptive gradient methods are among state of the art algorithms to train neural networks. The tuning of hyperparameters is needed to make them work efficiently…
math.NA2024
An Abstract Lyapunov Control Optimizer: Local Stabilization and Global Convergence
Bilel Bensaid, Gaël Poëtte, Rodolphe Turpault
Recently, many machine learning optimizers have been analysed considering them as the asymptotic limit of some differential equations when the step size goes to zero. In other word…