Automatic and Simultaneous Adjustment of Learning Rate and Momentum for Stochastic Gradient Descent

Tomer Lancewicki, Selcuk Kopru

Stochastic Gradient Descent (SGD) methods are prominent for training machine learning and deep learning models. The performance of these techniques depends on their hyperparameter…