Safe and Sharp Honest Inference for Nonparametric Estimation via Empirical Bernstein Calibration

Abstract

Honest confidence intervals for nonparametric estimators usually need to balance two competing goals: uniformly small undercoverage over a prescribed smoothness class and interval length of the minimax order. This paper develops empirical Bernstein confidence intervals (EBCIs), a calibration method that replaces standard-normal critical values with finite-sample Bernstein tail control. We first show that standard-normal calibration after bias correction faces two structural limitations: residual normalized bias can induce non-negligible undercoverage, and reducing stochastic variability can worsen coverage when the residual bias is fixed. EBCIs address these issues by combining a data-driven variance proxy, finite-grid bandwidth selection, and fixed-radius bias-aware optimization. For scalar-covariate nonparametric regression and density estimation, the proposed one- and two-sided intervals attain coverage at least $1-\hat{I}\pm$, uniformly over local $S$-smoothness classes, up to remainders of order $o(n^{-\frac{2S}{2S+1}})$, while their radii shrink at the minimax rate $n^{-\frac{S}{2S+1}}$. In sub-Gaussian settings, the order of the coverage error is exponentially small. We also demonstrate that EBCIs complement existing bias-aware procedures. They can be combined with robust bias correction, and they provide finite-sample coverage robustness in settings where fixed-length confidence intervals (FLCIs) may under-cover despite exact worst-case bias information. Simulations support the theoretical findings.