High-dimensional variable selection via low-dimensional adaptive learning

Abstract

A stochastic search method, the so-called Adaptive Subspace (AdaSub) method, is proposed for variable selection in high-dimensional linear regression models. The method aims at finding the best model with respect to a certain model selection criterion and is based on the idea of adaptively solving low-dimensional sub-problems in order to provide a solution to the original high-dimensional problem. Any of the usual ${\ell }_0$-type model selection criteria can be used, such as Akaike's Information Criterion (AIC), the Bayesian Information Criterion (BIC) or the Extended BIC (EBIC), with the last being particularly suitable for high-dimensional cases. The limiting properties of the new algorithm are analysed and it is shown that, under certain conditions, AdaSub converges to the best model according to the considered criterion. In a simulation study, the performance of AdaSub is investigated in comparison to alternative methods. The effectiveness of the proposed method is illustrated via various simulated datasets and a high-dimensional real data example.

References in corpus (26)

• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• High-dimensional graphs and variable selection with the Lasso
• Pathwise coordinate optimization
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Extended Bayesian Information Criteria for Gaussian Graphical Models
• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• Least squares after model selection in high-dimensional sparse models
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Variable selection with error control: Another look at Stability Selection
• Tuning parameter selection in high dimensional penalized likelihood
• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm
• Model Selection Principles in Misspecified Models
• High-dimensional variable selection via tilting
• When do stepwise algorithms meet subset selection criteria?
• Extensions of stability selection using subsamples of observations and covariates
• DECOrrelated feature space partitioning for distributed sparse regression

Cited by in corpus (2)

• Feature Selection for Huge Data via Minipatch Learning
• A Metropolized adaptive subspace algorithm for high-dimensional Bayesian variable selection