Gradient Descent's Last Iterate is Often (slightly) Suboptimal

Guy Kornowski, Ohad Shamir

We consider the well-studied setting of minimizing a convex Lipschitz function using either gradient descent (GD) or its stochastic variant (SGD), and examine the last iterate conv…