Deterministic Adam-Inspired Methods with Accelerated Convergence Rate

Yaxin Yu, Long Chen, Zeyi Xu

Adam is widely used, but its convergence theory remains incomplete even in the deterministic full-batch setting because momentum and adaptive preconditioning are tightly coupled. F…