Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression

Jingfeng Wu, Pierre Marion, Peter Bartlett

We study gradient descent (GD) with a constant stepsize for ${\ell }_2$-regularized logistic regression with linearly separable data. Classical theory suggests small stepsizes to ensu…