Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification

Bo Yang Hsueh, Wei Li, I-Chen Wu

Learning rate scheduler has been a critical issue in the deep neural network training. Several schedulers and methods have been proposed, including step decay scheduler, adaptive m…