Stochastic Gradient Descent for Operator Learning in Hilbert Spaces: Convergence Rates and Minimax Lower Bounds

Abstract

This study investigates the use of stochastic gradient descent (SGD) to learn operators between general Hilbert spaces. We study weak and strong regularity conditions for the target operator that characterize its structure and complexity. Under these conditions, we establish upper bounds for convergence rates of the SGD algorithm and derive a minimax lower bound analysis, further illustrating that our convergence analysis and regularity conditions quantitatively characterize the statistical difficulty of operator estimation under these regularity conditions. The analysis extends to nonlinear regression targets under model misspecification, in which case SGD converges to the best linear approximation. Moreover, applying our analysis to operator learning problems based on vector-valued and scalar-valued reproducing kernel Hilbert spaces yields new convergence results, thereby refining the conclusions of existing literature.

62 pages