On the Depth Transfer of Neural Optimizers via L-Smoothness

Maximilian Wong, Chenyu Xu, Jiajin Li +1

Scaling deep learning models efficiently requires predictable performance transfer across compute regimes. Establishing reliable hyperparameter transfer allows optimal training dyn…