An Adaptive and Momental Bound Method for Stochastic Learning

Jianbang Ding, Xuancheng Ren, Ruixuan Luo +1

Training deep neural networks requires intricate initialization and careful selection of learning rates. The emergence of stochastic gradient optimization methods that use adaptive…