A Finite Data Based Regularization in a Normed Linear Space Setting

Abstract

One of the basic problems in mathematical learning theory is to identify a function $f:\Omega \to \mathbb{R}$ with certain specific properties which fits a given training data $\{(x_i,{\xi }_i)\in \Omega \times \mathbb{R}:i=1,\dots ,n\}$, in the sense that, $f(x_i)={\xi }_i$ for $i=1,\dots ,n$, where $\Omega$ is a compact subset of ${\mathbb{R}}^d$ for some $d\in \mathbb{N}$ and $f$ is required to have certain specified characteristics. We address this problem when $f$ belongs to an arbitrary normed linear space $X$, and the evaluation maps $f\mapsto f(x_i)$ are replaced by maps of the form $f\mapsto {\phi }_i(f)$ on $X$, where ${\phi }_1,\dots ,{\phi }_n$ are continuous linear functionals on $X$. Using an inner product structure on ${\mathbb{R}}^n$, we shall device a method of least-squares for obtaining an approximate solution for the above problem and also identify a subspace of $X$ in which the least-square solution is unique, which in turn is shown to be equivalent to solving a matrix equation. In the context when the matrix under consideration is ill-conditioned, a regularized equation is devised, and order optimal error estimates are derived for the exact targeted data $\xi =({\xi }_1,\dots ,{\xi }_n)$ and also when it is noisy, by choosing the regularization parameter appropriately.