Training Deep Neural Networks by optimizing over nonlocal paths in hyperparameter space

Vlad Pushkarov, Jonathan Efroni, Mykola Maksymenko +1

Hyperparameter optimization is both a practical issue and an interesting theoretical problem in training of deep architectures. Despite many recent advances the most commonly used…