Where Do Large Learning Rates Lead Us?

Ildus Sadrtdinov, Maxim Kodryan, Eduard Pokonechny +2

It is generally accepted that starting neural networks training with large learning rates (LRs) improves generalization. Following a line of research devoted to understanding this…