MADA: Meta-Adaptive Optimizers through hyper-gradient Descent

Kaan Ozkara, Can Karakus, Parameswaran Raman +4

Following the introduction of Adam, several novel adaptive optimizers for deep learning have been proposed. These optimizers typically excel in some tasks but may not outperform Ad…