Statistical Learning Theory for Neural Operators

Abstract

We present statistical convergence results for the learning of (possibly) non-linear mappings in infinite-dimensional spaces. Specifically, given a map $G_0:\mathcal{X}\to \mathcal{Y}$ between two separable Hilbert spaces, we analyze the problem of recovering $G_0$ from $n\in \mathbb{N}$ noisy input-output pairs $(x_i,y_i{)}_{i=1}^n$ with $y_i=G_0(x_i)+{\epsilon }_i$; here the $x_i\in \mathcal{X}$ represent randomly drawn 'design' points, and the ${\epsilon }_i$ are assumed to be either i.i.d. white noise processes or subgaussian random variables in $\mathcal{Y}$. We provide general convergence results for least-squares-type empirical risk minimizers over compact regression classes $\mathbf{G}\subseteq L^{\infty }(X,Y)$, in terms of their approximation properties and metric entropy bounds, which are derived using empirical process techniques. This generalizes classical results from finite-dimensional nonparametric regression to an infinite-dimensional setting. As a concrete application, we study an encoder-decoder based neural operator architecture termed FrameNet. Assuming $G_0$ to be holomorphic, we prove algebraic (in the sample size $n$) convergence rates in this setting, thereby overcoming the curse of dimensionality. To illustrate the wide applicability, as a prototypical example we discuss the learning of the non-linear solution operator to a parametric elliptic partial differential equation.