UPS delivers optimal phase diagram in high-dimensional variable selection

Abstract

Consider a linear model $Y=X\beta +z$, $z\sim N(0,I_n)$. Here, $X=X_{n,p}$, where both $p$ and $n$ are large, but $p>n$. We model the rows of $X$ as i.i.d. samples from $N(0,\frac{1}{n}\Omega )$, where $\Omega$ is a $p\times p$ correlation matrix, which is unknown to us but is presumably sparse. The vector $\beta$ is also unknown but has relatively few nonzero coordinates, and we are interested in identifying these nonzeros. We propose the Univariate Penalization Screeing (UPS) for variable selection. This is a screen and clean method where we screen with univariate thresholding and clean with penalized MLE. It has two important properties: sure screening and separable after screening. These properties enable us to reduce the original regression problem to many small-size regression problems that can be fitted separately. The UPS is effective both in theory and in computation.

Published in at http://dx.doi.org/10.1214/11-AOS947 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (4)

• Covariance regularization by thresholding
• High-dimensional variable selection
• Near-ideal model selection by ${\ell }_1$ minimization
• Thresholded Lasso for high dimensional variable selection and statistical estimation

Cited by in corpus (5)

• Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects
• Covariate assisted screening and estimation
• Signal Identification for Rare and Weak Features: Higher Criticism or False Discovery Rates?
• Optimal classification in sparse Gaussian graphic model
• KL-BSS: Rethinking optimality for neighbourhood selection in structural equation models