Memory Augmented Optimizers for Deep Learning

Paul-Aymeric McRae, Prasanna Parthasarathi, Mahmoud Assran +1

Popular approaches for minimizing loss in data-driven learning often involve an abstraction or an explicit retention of the history of gradients for efficient parameter updates. Th…