On the properties of the linear conjugate gradient method

Abstract

The linear conjugate gradient method is an efficient iterative method for the convex quadratic minimization problems $\underset{x\in {\mathbb{R}}^n}{\min }f(x)=\frac{1}{2}{x}^{T}Ax+b^{T}x$, where $A\in R^{n\times n}$ is symmetric and positive definite and $b\in R^n$. It is generally agreed that the gradients $g_k$ are not conjugate with respective to $A$ in the linear conjugate gradient method (see page 111 in Numerical optimization (2nd, Springer, 2006) by Nocedal and Wright). In the paper we prove the conjugacy of the gradients $g_k$ generated by the linear conjugate gradient method, namely, $$g_k^{T}A{g}_i=0,i=0,1,\cdots ,k-2.$$ In addition,a new way is exploited to derive the linear conjugate gradient method based on the conjugacy of the search directions and the orthogonality of the gradients, rather than the conjugacy of the search directions and the exact stepsize.