Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf

Priya Kasimbeg, Ahmed Khaled, Aahlad Chandrabhatta +9

Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure true algorithmic performance…