4 papers
Using a one dimensional parabolic model of the full-batch loss to estimate learning rates during training
Maximus Mutschler, Kevin Laube, Andreas Zell
A fundamental challenge in Deep Learning is to find optimal step sizes for stochastic gradient descent automatically. In traditional optimization, line searches are a commonly used…
Empirically explaining SGD from a line search perspective
Maximus Mutschler, Andreas Zell
Optimization in Deep Learning is mainly guided by vague intuitions and strong assumptions, with a limited understanding how and why these work in practice. To shed more light on th…
A straightforward line search approach on the expected empirical loss for stochastic deep learning problems
Maximus Mutschler, Andreas Zell
A fundamental challenge in deep learning is that the optimal step sizes for update steps of stochastic gradient descent are unknown. In traditional optimization, line searches are…
Parabolic Approximation Line Search for DNNs
Maximus Mutschler, Andreas Zell
A major challenge in current optimization research for deep learning is to automatically find optimal step sizes for each update step. The optimal step size is closely related to t…