MSP: A Multi-step Screening Procedure for Sparse Recovery

Abstract

We propose a Multi-step Screening Procedure (MSP) for the recovery of sparse linear models in high-dimensional data. This method is based on a repeated small penalty strategy that quickly converges to an estimate within a few iterations. Specifically, in each iteration, an adaptive lasso regression with a small penalty is fit within the reduced feature space obtained from the previous step, rendering its computational complexity roughly comparable with the Lasso. MSP is shown to select the true model under complex correlation structures among the predictors and response, even when the irrepresentable condition fails. Further, under suitable regularity conditions, MSP achieves the optimal minimax rate $(q\log n/n{)}^{1/2}$ for the upper bound of $l_2$-norm error. Numerical comparisons show that the method works effectively both in model selection and estimation, and the MSP fitted model is stable over a range of small tuning parameter values, eliminating the need to choose the tuning parameter by cross-validation. We also apply MSP to financial data and show that MSP is successful in asset allocation selection.

References in corpus (15)

• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Nearly unbiased variable selection under minimax concave penalty
• Nonconcave penalized likelihood with a diverging number of parameters
• Strong oracle optimality of folded concave penalized estimation
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Global solutions to folded concave penalized nonconvex learning