Automated Learning Rate Scheduler for Large-batch Training

Chiheon Kim, Saehoon Kim, Jongmin Kim +2

Large-batch training has been essential in leveraging large-scale datasets and models in deep learning. While it is computationally beneficial to use large batch sizes, it often re…