Optimizers Qualitatively Alter Solutions And We Should Leverage This

Razvan Pascanu, Clare Lyle, Ionut-Vlad Modoranu +6

Due to the nonlinear nature of Deep Neural Networks (DNNs), one can not guarantee convergence to a unique global minimum of the loss when using optimizers relying only on local inf…