Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables
arXiv:2106.09071
Abstract
Strong correlations between explanatory variables are problematic for high-dimensional regularized regression methods. Due to the violation of the Irrepresentable Condition, the popular LASSO method may suffer from false inclusions of inactive variables. In this paper, we propose pre-processing with orthogonal decompositions (PROD) for the explanatory variables in high-dimensional regressions. The PROD procedure is constructed based upon a generic orthogonal decomposition of the design matrix. We demonstrate by two concrete cases that the PROD approach can be effectively constructed for improving the performance of high-dimensional penalized regression. Our theoretical analysis reveals their properties and benefits for high-dimensional penalized linear regression with LASSO. Extensive numerical studies with simulations and data analysis show the promising performance of the PROD.
References in corpus (7)
- Nearly unbiased variable selection under minimax concave penalty
- Challenges of Big Data Analysis
- On the adaptive elastic-net with a diverging number of parameters
- Asymptotic properties of bridge estimators in sparse high-dimensional regression models
- High-dimensional covariance matrix estimation in approximate factor models
- "Pre-conditioning" for feature selection and regression in high-dimensional problems
- Covariate assisted screening and estimation