Empirical Risk Minimization as Parameter Choice Rule for General Linear Regularization Methods

Abstract

We consider the statistical inverse problem to recover $f$ from noisy measurements $Y=Tf+\sigma \xi$ where $\xi$ is Gaussian white noise and $T$ a compact operator between Hilbert spaces. Considering general reconstruction methods of the form ${\hat{f}}_{\alpha }=q_{\alpha }\left(T^{\ast }T\right)T^{\ast }Y$ with an ordered filter $q_{\alpha }$, we investigate the choice of the regularization parameter $\alpha$ by minimizing an unbiased estimate of the predictive risk $\mathbb{E}\left[\Vert Tf-T{\hat{f}}_{\alpha }{\Vert }^2\right]$. The corresponding parameter ${\alpha }_{\mathrm{pred}}$ and its usage are well-known in the literature, but oracle inequalities and optimality results in this general setting are unknown. We prove a (generalized) oracle inequality, which relates the direct risk $\mathbb{E}\left[\Vert f-{\hat{f}}_{{\alpha }_{\mathrm{pred}}}{\Vert }^2\right]$ with the oracle prediction risk ${\inf }_{\alpha >0}\mathbb{E}\left[\Vert Tf-T{\hat{f}}_{\alpha }{\Vert }^2\right]$. From this oracle inequality we are then able to conclude that the investigated parameter choice rule is of optimal order. Finally we also present numerical simulations, which support the order optimality of the method and the quality of the parameter choice in finite sample situations.

References in corpus (1)

• Risk hull method and regularization by projections of ill-posed inverse problems

Cited by in corpus (6)

• A modified discrepancy principle to attain optimal convergence rates under unknown noise
• A Variational View on Statistical Multiscale Estimation
• Predictive risk estimation for the Expectation Maximization algorithm with Poisson data
• Adaptive minimax optimality in statistical inverse problems via SOLIT -- Sharp Optimal Lepskii-Inspired Tuning
• On the asymptotical regularization for linear inverse problems in presence of white noise
• Unbiased Bregman-Risk Estimators: Application to Regularization Parameter Selection in Tomographic Image Reconstruction