Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning

Hoang Tran, Qinzi Zhang, Ashok Cutkosky

There is a significant gap between our theoretical understanding of optimization algorithms used in deep learning and their practical performance. Theoretical development usually f…