AutoDrop: Training Deep Learning Models with Automatic Learning Rate Drop

Yunfei Teng, Jing Wang, Anna Choromanska

Modern deep learning (DL) architectures are trained using variants of the SGD algorithm that is run with a $\text{manually}$ defined learning rate schedule, i.e., the learning ra…