Convergence rates of stochastic gradient method with independent sequences of step-size and momentum weight

Wen-Liang Hwang

In large-scale learning algorithms, the momentum term is usually included in the stochastic sub-gradient method to improve the learning speed because it can navigate ravines effici…