Improved Concentration for Mean Estimators via Shrinkage

Abstract

We study a class of robust mean estimators $\hat{\hat{I}}¼$ obtained by adaptively shrinking the weights of sample points far from a base estimator $\hat{\hat{I}}º$. Given a data-dependent scaling factor $\hat{\hat{I}}\pm$ and a weighting function $w:[0,\infty )\to [0,1]$, we let $\hat{\hat{I}}¼=\hat{\hat{I}}º+\frac{1}{n}{\sum }_{i=1}^n(X_i-\hat{\hat{I}}º)w(\hat{\hat{I}}\pm |X_i-\hat{\hat{I}}º|)$. We prove that, under mild assumptions over $w$, these estimators achieve stronger concentration bounds than the base estimate $\hat{\hat{I}}º$, including sub-Gaussian guarantees. This framework unifies and extends several existing approaches to robust mean estimation in $\mathbb{R}$, and can also be generalized to the multivariate setting. Through numerical experiments, we show that our shrinking approach translates to faster concentration, even for small sample sizes.

35 pages, 3 figures