Convergence Analysis of Greedy Algorithms with Adaptive Relaxation in Hilbert Spaces

Abstract

The Power-Relaxed Greedy Algorithm (PRGA) was introduced as a generalization of the so called Relaxed Greedy Algorithm, introduced by DeVore and Temlyakov, by replacing the relaxation parameter $1/m$ with $1/m^{\hat{I}}\pm$, with the aim of improving convergence rates. While the case $\hat{I}\pm \le 1$ is well understood, the behavior of the algorithm for $\hat{I}\pm >1$ remained an open problem. In this work, we answer this question and, moreover, we introduce a relaxed greedy algorithm with an optimal step size chosen by exact line search at each iteration.