Understanding and correcting pathologies in the training of learned optimizers

Luke Metz, Niru Maheswaranathan, Jeremy Nixon +2

Deep learning has shown that learned functions can dramatically outperform hand-designed functions on perceptual tasks. Analogously, this suggests that learned optimizers may simil…