A family of spectral gradient methods for optimization
arXiv:1812.02974
Abstract
We propose a family of spectral gradient methods, whose stepsize is determined by a convex combination of the long Barzilai-Borwein (BB) stepsize and the short BB stepsize. Each member of the family is shown to share certain quasi-Newton property in the sense of least squares. The family also includes some other gradient methods as its special cases. We prove that the family of methods is -superlinearly convergent for two-dimensional strictly convex quadratics. Moreover, the family is -linearly convergent in the any-dimensional case. Numerical results of the family with different settings are presented, which demonstrate that the proposed family is promising.
22 pages, 2figures