Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond
Bohan Wang, Huishuai Zhang, Qi Meng +3
This paper aims to clearly distinguish between Stochastic Gradient Descent with Momentum (SGDM) and Adam in terms of their convergence rates. We demonstrate that Adam achieves a fa…
cs.LG2023
Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity
Bohan Wang, Jingwen Fu, Huishuai Zhang +2
Recently, Arjevani et al. [1] established a lower bound of iteration complexity for the first-order optimization under an -smooth condition and a bounded noise variance assumpti…