Exploring loss function topology with cyclical learning rates

Leslie N. Smith, Nicholay Topin

We present observations and discussion of previously unreported phenomena discovered while training residual networks. The goal of this work is to better understand the nature of n…