Investigation into the Training Dynamics of Learned Optimizers

Jan Sobotka, Petr Šimánek, Daniel Vašata

Optimization is an integral part of modern deep learning. Recently, the concept of learned optimizers has emerged as a way to accelerate this optimization process by replacing trad…