No More Pesky Learning Rates

Tom Schaul, Sixin Zhang, Yann LeCun

The performance of stochastic gradient descent (SGD) depends critically on how learning rates are tuned and decreased over time. We propose a method to automatically adjust multipl…