Adaptive robust variable selection

Abstract

Heavy-tailed high-dimensional data are commonly encountered in various scientific fields and pose great challenges to modern statistical analysis. A natural procedure to address this problem is to use penalized quantile regression with weighted $L_1$-penalty, called weighted robust Lasso (WR-Lasso), in which weights are introduced to ameliorate the bias problem induced by the $L_1$-penalty. In the ultra-high dimensional setting, where the dimensionality can grow exponentially with the sample size, we investigate the model selection oracle property and establish the asymptotic normality of the WR-Lasso. We show that only mild conditions on the model error distribution are needed. Our theoretical results also reveal that adaptive choice of the weight vector is essential for the WR-Lasso to enjoy these nice asymptotic properties. To make the WR-Lasso practically feasible, we propose a two-step procedure, called adaptive robust Lasso (AR-Lasso), in which the weight vector in the second step is constructed based on the $L_1$-penalized quantile regression estimate from the first step. This two-step procedure is justified theoretically to possess the oracle property and the asymptotic normality. Numerical studies demonstrate the favorable finite-sample performance of the AR-Lasso.

arXiv commentPublished in at http://dx.doi.org/10.1214/13-AOS1191 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (7)

• Nearly unbiased variable selection under minimax concave penalty
• Simultaneous analysis of Lasso and Dantzig selector
• Nonconcave penalized likelihood with a diverging number of parameters
• Composite quantile regression and the oracle Model Selection Theory
• A unified approach to model selection and sparse recovery using regularized least squares
• Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
• Quasi-Likelihood and/or Robust Estimation in High Dimensions

Cited by in corpus (8)

• Strong oracle optimality of folded concave penalized estimation
• Oracle Estimation of a Change Point in High Dimensional Quantile Regression
• Robust Variable Selection and Estimation Via Adaptive Elastic Net S-Estimators for Linear Regression
• Fast and Robust Sparsity Learning over Networks: A Decentralized Surrogate Median Regression Approach
• False Discovery Rate Control for High-Dimensional Networks of Quantile Associations Conditioning on Covariates
• Predictive Quantile Regression with Mixed Roots and Increasing Dimensions: The ALQR Approach
• Efficient Sparse Least Absolute Deviation Regression with Differential Privacy
• Wilcoxon-type Multivariate Cluster Elastic Net