Improved Variance Estimation in Homoskedastic Nonparametric Random-Design Regression via a Two-Scale Approach

Abstract

We study estimation of a constant conditional variance ${\sigma }^2$ in nonparametric regression with a $d$-dimensional random design. This is an important problem, and similar questions arise in causal inference. The regression function is ${\beta }_b$-Hölder smooth, the design density is ${\beta }_g$-Hölder smooth and bounded above and away from zero, and we consider the nonparametric regime ${\beta }_b>1$ and $d>4{\beta }_b$. Set ${\beta }_g^{\star }={\beta }_b(1-4{\beta }_b/d)/\{1+2{\beta }_b/d+8({\beta }_b/d{)}^2\}$. We give an estimator whose mean squared error is upper bounded by $C{n}^{-4({\beta }_b+1)/(d+4)}$ in the low-regularity regime when $0<{\beta }_g\le {\beta }_g^{\star }$. The low-regularity branch is based on a new two-scale construction: the covariate space is partitioned into cells, the local polynomial trend is projected out within each suitable cell, and the squared normalized contrast from one eligible close pair per cell is averaged across cells. In the high regularity regime when ${\beta }_g>{\beta }_g^{\star }$, a higher-order influence function estimator of Robins, Li, Tchetgen Tchetgen, and van der Vaart (2008) provides the rate $C{n}^{-8{\beta }_b/(d+4{\beta }_b)}$. We also give an all-pairs ridge extension, which achieves the same two-scale rate, and evaluate the methods alongside a range of existing estimators in simulations.