185 citations · 286 across the 34 of their papers we have counts for
7 papers · 1 filter
Using a one dimensional parabolic model of the full-batch loss to estimate learning rates during training
Maximus Mutschler, Kevin Laube, Andreas Zell
A fundamental challenge in Deep Learning is to find optimal step sizes for stochastic gradient descent automatically. In traditional optimization, line searches are a commonly used…
Conditional super-network weights
Kevin Alexander Laube, Andreas Zell
Modern Neural Architecture Search methods have repeatedly broken state-of-the-art results for several disciplines. The super-network, a central component of many such methods, enab…
Empirically explaining SGD from a line search perspective
Maximus Mutschler, Andreas Zell
Optimization in Deep Learning is mainly guided by vague intuitions and strong assumptions, with a limited understanding how and why these work in practice. To shed more light on th…
A straightforward line search approach on the expected empirical loss for stochastic deep learning problems
Maximus Mutschler, Andreas Zell
A fundamental challenge in deep learning is that the optimal step sizes for update steps of stochastic gradient descent are unknown. In traditional optimization, line searches are…
Prune and Replace NAS
Kevin Alexander Laube, Andreas Zell
While recent NAS algorithms are thousands of times faster than the pioneering works, it is often overlooked that they use fewer candidate operations, resulting in a significantly s…
Parabolic Approximation Line Search for DNNs
Maximus Mutschler, Andreas Zell
A major challenge in current optimization research for deep learning is to automatically find optimal step sizes for each update step. The optimal step size is closely related to t…
ShuffleNASNets: Efficient CNN models through modified Efficient Neural Architecture Search
Kevin Alexander Laube, Andreas Zell
Neural network architectures found by sophistic search algorithms achieve strikingly good test performance, surpassing most human-crafted network models by significant margins. Alt…