Efficient Long-Horizon Learning for Learned Optimization

Xiaolong Huang, Benjamin Thérien, James Harrison +1

Learned optimization aims to improve upon hand-designed optimizers (e.g., Adam and Muon) by meta-learning small neural network optimizers over a distribution of tasks. While recent…