An Exponentially Increasing Step-size for Parameter Estimation in Statistical Models

Nhat Ho, Tongzheng Ren, Sujay Sanghavi +2

Using gradient descent (GD) with fixed or decaying step-size is a standard practice in unconstrained optimization problems. However, when the loss function is only locally convex,…