Showing 2021Show all
2 papers · 1 filter
cs.LG2021
Using a one dimensional parabolic model of the full-batch loss to estimate learning rates during training
Maximus Mutschler, Kevin Laube, Andreas Zell
A fundamental challenge in Deep Learning is to find optimal step sizes for stochastic gradient descent automatically. In traditional optimization, line searches are a commonly used…
cs.LG2021
Empirically explaining SGD from a line search perspective
Maximus Mutschler, Andreas Zell
Optimization in Deep Learning is mainly guided by vague intuitions and strong assumptions, with a limited understanding how and why these work in practice. To shed more light on th…