Cumulative Learning Rate Adaptation: Revisiting Path-Based Schedules for SGD and Adam

Asma Atamna, Tom Maus, Fabian Kievelitz +1

The learning rate is a crucial hyperparameter in deep learning, with its ideal value depending on the problem and potentially changing during training. In this paper, we investigat…