Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity

Bohan Wang, Jingwen Fu, Huishuai Zhang +2

Recently, Arjevani et al. [1] established a lower bound of iteration complexity for the first-order optimization under an $L$-smooth condition and a bounded noise variance assumpti…