822 citations · 899 across the 14 of their papers we have counts for
Showing 2017Show all
3 papers · 1 filter
cs.LG2017
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
Leslie N. Smith, Nicholay Topin
In this paper, we describe a phenomenon, which we named "super-convergence", where neural networks can be trained an order of magnitude faster than with standard training methods.…
cs.SE2017★ 4 cited
Best Practices for Applying Deep Learning to Novel Applications
Leslie N. Smith
This report is targeted to groups who are subject matter experts in their application but deep learning novices. It contains practical advice for those interested in testing the us…
cs.LG2017★ 18 cited
Exploring loss function topology with cyclical learning rates
Leslie N. Smith, Nicholay Topin
We present observations and discussion of previously unreported phenomena discovered while training residual networks. The goal of this work is to better understand the nature of n…