The Loss Rank Criterion for Variable Selection in Linear Regression Analysis
arXiv:1011.1373 · doi:10.1111/j.1467-9469.2011.00732.x
Abstract
Lasso and other regularization procedures are attractive methods for variable selection, subject to a proper choice of shrinkage parameter. Given a set of potential subsets produced by a regularization algorithm, a consistent model selection criterion is proposed to select the best one among this preselected set. The approach leads to a fast and efficient procedure for variable selection, especially in high-dimensional settings. Model selection consistency of the suggested criterion is proven when the number of covariates d is fixed. Simulation studies suggest that the criterion still enjoys model selection consistency when d is much larger than the sample size. The simulations also show that our approach for variable selection works surprisingly well in comparison with existing competitors. The method is also applied to a real data set.
arXiv comment18 pages, 1 figure
References in corpus (15)
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- High-dimensional graphs and variable selection with the Lasso
- On the "degrees of freedom" of the lasso
- Testing the order of a model
- On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding
- Model Selection by Loss Rank for Classification and Unsupervised Learning