Better Convergence Guarantees for Sign-Based Momentum Methods

Wei Jiang, Dingzhi Yu, Sifan Yang +3

This paper presents an improved analysis for sign-based methods with momentum updates. Traditional sign-based methods obtain a convergence rate of $\mathcal{O}(T^{-1/4})$ under the…