High-dimensional variable selection via tilting

Abstract

The paper considers variable selection in linear regression models where the number of covariates is possibly much larger than the number of observations. High dimensionality of the data brings in many complications, such as (possibly spurious) high correlations between the variables, which result in marginal correlation being unreliable as a measure of association between the variables and the response. We propose a new way of measuring the contribution of each variable to the response which takes into account high correlations between the variables in a data-driven way. The proposed tilting procedure provides an adaptive choice between the use of marginal correlation and tilted correlation for each variable, where the choice is made depending on the values of the hard thresholded sample correlation of the design matrix. We study the conditions under which this measure can successfully discriminate between the relevant and the irrelevant variables and thus be used as a tool for variable selection. Finally, an iterative variable screening algorithm is constructed to exploit the theoretical properties of tilted correlation, and its good practical performance is demonstrated in a comparative simulation study.

References in corpus (19)

• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Nearly unbiased variable selection under minimax concave penalty
• Covariance regularization by thresholding
• Estimating high-dimensional directed acyclic graphs with the PC-algorithm
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm
• Improved variable selection with Forward-Lasso adaptive shrinkage
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)

Cited by in corpus (14)

• Interaction Pursuit with Feature Screening and Selection
• Which bridge estimator is optimal for variable selection?
• High-dimensional variable selection via low-dimensional adaptive learning
• No penalty no tears: Least squares in high-dimensional linear models
• On the consistency theory of high dimensional variable screening
• Generalized Fiducial Inference for Ultrahigh Dimensional Regression
• Bayesian iterative screening in ultra-high dimensional linear regressions
• Robust variable screening for regression using factor profiling
• Variable Selection for Survival Data with A Class of Adaptive Elastic Net Techniques
• Ultra High Dimensional Change Point Detection
• High-dimensional Ordinary Least-squares Projection for Screening Variables
• Exploring dependence between categorical variables: benefits and limitations of using variable selection within Bayesian clustering in relation to log-linear modelling with interaction terms
• Safe preselection in lasso-type problems by cross-validation freezing
• Covariance-Insured Screening