A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)

Abstract

High dimensional statistical problems arise from diverse fields of scientific research and technological development. Variable selection plays a pivotal role in contemporary statistical learning and scientific discoveries. The traditional idea of best subset selection methods, which can be regarded as a specific form of penalized likelihood, is computationally too expensive for many modern statistical applications. Other forms of penalized likelihood methods have been successfully developed over the last decade to cope with high dimensionality. They have been widely applied for simultaneously selecting important variables and estimating their effects in high dimensional statistical inference. In this article, we present a brief account of the recent developments of theory, methods, and implementations for high dimensional variable selection. What limits of the dimensionality such methods can handle, what the role of penalty functions is, and what the statistical properties are rapidly drive the advances of the field. The properties of non-concave penalized likelihood and its roles in high dimensional statistical modeling are emphasized. We also review some recent advances in ultra-high dimensional variable selection, with emphasis on independence screening and two-scale methods.

To appear in Statistica Sinica, 44 pages, 5 figures

References in corpus (18)

• Simultaneous analysis of Lasso and Dantzig selector
• Pathwise coordinate optimization
• Nonconcave penalized likelihood with a diverging number of parameters
• Sure independence screening in generalized linear models with NP-dimensionality
• Composite quantile regression and the oracle Model Selection Theory
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Piecewise linear regularized solution paths
• High-dimensional classification using features annealed independence rules
• Choice of neighbor order in nearest-neighbor classification
• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery
• Sharp thresholds for high-dimensional and noisy recovery of sparsity
• Robustness of multiple testing procedures against dependence
• Discussion: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Sparse inverse covariance estimation with the lasso
• Asymptotic minimaxity of False Discovery Rate thresholding for sparse exponential data
• Discussion: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Ultrahigh dimensional variable selection: beyond the linear model
• Sparse recovery in convex hulls via entropy penalization

Cited by in corpus (8)

• High-dimensional variable selection via tilting
• Are Discoveries Spurious? Distributions of Maximum Spurious Correlations and Their Applications
• Asymptotic properties for combined $L_1$ and concave regularization
• RANK: Large-Scale Inference with Graphical Nonlinear Knockoffs
• An Algorithm for Nonlinear, Nonparametric Model Choice and Prediction
• Sharp Support Recovery from Noisy Random Measurements by L1 minimization
• Large-Scale Model Selection with Misspecification
• Structure learning via unstructured kernel-based M-regression