14 citations · 14 across the 2 of their papers we have counts for
2 papers
cs.LG2021
LRTuner: A Learning Rate Tuner for Deep Neural Networks
Nikhil Iyer, V Thejas, Nipun Kwatra +2
One very important hyperparameter for training deep neural networks is the learning rate schedule of the optimizer. The choice of learning rate schedule determines the computationa…
cs.LG2020★ 14 cited
Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedule
Nikhil Iyer, V Thejas, Nipun Kwatra +2
Several papers argue that wide minima generalize better than narrow minima. In this paper, through detailed experiments that not only corroborate the generalization properties of w…