Asymptotic normality of robust $M$-estimators with convex penalty

Abstract

This paper develops asymptotic normality results for individual coordinates of robust M-estimators with convex penalty in high-dimensions, where the dimension $p$ is at most of the same order as the sample size $n$, i.e, $p/n\le \gamma$ for some fixed constant $\gamma >0$. The asymptotic normality requires a bias correction and holds for most coordinates of the M-estimator for a large class of loss functions including the Huber loss and its smoothed versions regularized with a strongly convex penalty. The asymptotic variance that characterizes the width of the resulting confidence intervals is estimated with data-driven quantities. This estimate of the variance adapts automatically to low ($p/n\to 0)$ or high ($p/n\le \gamma$) dimensions and does not involve the proximal operators seen in previous works on asymptotic normality of M-estimators. For the Huber loss, the estimated variance has a simple expression involving an effective degrees-of-freedom as well as an effective sample size. The case of the Huber loss with Elastic-Net penalty is studied in details and a simulation study confirms the theoretical findings. The asymptotic normality results follow from Stein formulae for high-dimensional random vectors on the sphere developed in the paper which are of independent interest.

References in corpus (8)

• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Statistical significance in high-dimensional linear models
• A framework to characterize performance of LASSO algorithms
• Asymptotic behavior of unregularized and ridge-regularized high-dimensional robust regression estimators : rigorous results
• The Lasso with general Gaussian designs with applications to hypothesis testing
• De-Biasing The Lasso With Degrees-of-Freedom Adjustment
• Second order Stein: SURE for SURE and other applications in high-dimensional inference
• Out-of-sample error estimate for robust M-estimators with convex penalty