A straightforward line search approach on the expected empirical loss for stochastic deep learning problems

Maximus Mutschler, Andreas Zell

A fundamental challenge in deep learning is that the optimal step sizes for update steps of stochastic gradient descent are unknown. In traditional optimization, line searches are…