Lower bounds for the trade-off between bias and mean absolute deviation

Abstract

In nonparametric statistics, rate-optimal estimators typically balance bias and stochastic error. The recent work on overparametrization raises the question whether rate-optimal estimators exist that do not obey this trade-off. In this work we consider pointwise estimation in the Gaussian white noise model with regression function $f$ in a class of ${\hat{I}}^2$-Hölder smooth functions. Let 'worst-case' refer to the supremum over all functions $f$ in the Hölder class. It is shown that any estimator with worst-case bias $\lesssim n^{-{\hat{I}}^2/(2{\hat{I}}^2+1)}=:\ddot{I}{ˆ}_n$ must necessarily also have a worst-case mean absolute deviation that is lower bounded by $\gtrsim \ddot{I}{ˆ}_n.$ To derive the result, we establish abstract inequalities relating the change of expectation for two probability measures to the mean absolute deviation.

This is an extended version of Section 7 of arXiv:2006.00278v3. The material has been removed from later versions of arXiv:2006.00278