Structured Spectral Step-Sizes and Hanoi-type Ordering for Gradient Methods

Abstract

Gradient methods are widely used for optimization, yet their practical convergence performance depends critically on step-size selection. Spectral step-size selection involves two interrelated questions: how to construct reliable candidates from iteration history, and how to order them during the iteration. This paper addresses both from a spectral viewpoint. For candidate construction, we develop a unified framework relating Huang-Dai-Liu (HDL) determinant pencils, limited-memory steepest descent (LMSD)-type Krylov-Ritz extraction with pseudo-memory realization, and Gu-Du moment recovery with component-energy weights. The framework shows that generalized-eigenvalue recoverability, bounded-window implementability, and energy interpretability coincide in a positive rank-one finite-moment regime, providing a structural criterion that explains why LMSD-type Ritz values form a natural low-memory candidate pool. For the complementary ordering question, we identify a rebound phenomenon in which low-frequency steps amplify higher-frequency components, leading to a recursive high-frequency-first ordering formulated as a Hanoi-type principle and extended to memory-m settings as a controlled scheduling rule. Based on these insights, we propose an adaptive gradient method with component-energy selection and phase-length control, prove global R-linear convergence for strictly convex quadratic objectives under the proposed admissibility filter and demonstrate competitive performance on ill-conditioned problems.

48 pages,6 figures