Ultrahigh dimensional variable selection: beyond the linear model

Abstract

Variable selection in high-dimensional space characterizes many contemporary problems in scientific discovery and decision making. Many frequently-used techniques are based on independence screening; examples include correlation ranking (Fan and Lv, 2008) or feature selection using a two-sample t-test in high-dimensional classification (Tibshirani et al., 2003). Within the context of the linear model, Fan and Lv (2008)showed that this simple correlation ranking possesses a sure independence screening property under certain conditions and that its revision, called iteratively sure independent screening (ISIS), is needed when the features are marginally unrelated but jointly related to the response variable. In this paper, we extend ISIS, without explicit definition of residuals, to a general pseudo-likelihood framework, which includes generalized linear models as a special case. Even in the least-squares setting, the new method improves ISIS by allowing variable deletion in the iterative process. Our technique allows us to select important features in high-dimensional classification where the popularly used two-sample t-method fails. A new technique is introduced to reduce the false discovery rate in the feature screening stage. Several simulated and two real data examples are presented to illustrate the methodology.

32 pages

References in corpus (13)

• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Simultaneous analysis of Lasso and Dantzig selector
• Nonconcave penalized likelihood with a diverging number of parameters
• "Pre-conditioning" for feature selection and regression in high-dimensional problems

Cited by in corpus (6)

• Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data
• Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• MM Algorithms for Minimizing Nonsmoothly Penalized Objective Functions
• RANK: Large-Scale Inference with Graphical Nonlinear Knockoffs
• Marginal and Interactive Feature Screening of Ultra-high Dimensional Feature Spaces with Multivariate Response