Iteratively Reweighted ${\ell }_1$-Penalized Robust Regression

Abstract

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, we show that iteratively reweighted ${\ell }_1$-penalized adaptive Huber regression estimator satisfies exponential deviation bounds and oracle properties, including the oracle convergence rate and variable selection consistency, under a weak beta-min condition. Computationally, we need as many as $O(\log s+\log \log d)$ iterations to reach such an oracle estimator, where $s$ and $d$ denote the sparsity and ambient dimension, respectively. Extension to a general class of robust loss functions is also considered. Numerical studies lend strong support to our methodology and theory.

arXiv comment62 pages

References in corpus (5)

• Nearly unbiased variable selection under minimax concave penalty
• Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming
• SLOPE - Adaptive variable selection via convex optimization
• Sparse Regression: Scalable algorithms and empirical performance
• Dimension-free PAC-Bayesian bounds for matrices, vectors, and linear least squares regression