Training With Data Dependent Dynamic Learning Rates

Shreyas Saxena, Nidhi Vyas, Dennis DeCoste

Recently many first and second order variants of SGD have been proposed to facilitate training of Deep Neural Networks (DNNs). A common limitation of these works stem from the fact…