Adaptive Learning Rate and Momentum for Training Deep Neural Networks

Zhiyong Hao, Yixuan Jiang, Huihua Yu +1

Recent progress on deep learning relies heavily on the quality and efficiency of training algorithms. In this paper, we develop a fast training method motivated by the nonlinear Co…