The Essential Best and Average Rate of Convergence of the Exact Line Search Gradient Descent Method

Thomas Yu

It is very well known that when the exact line search gradient descent method is applied to a convex quadratic objective, the worst-case rate of convergence (ROC), among all seed v…