Calibrating nonconvex penalized regression in ultra-high dimension

Abstract

We investigate high-dimensional nonconvex penalized regression, where the number of covariates may grow at an exponential rate. Although recent asymptotic theory established that there exists a local minimum possessing the oracle property under general conditions, it is still largely an open problem how to identify the oracle estimator among potentially multiple local minima. There are two main obstacles: (1) due to the presence of multiple minima, the solution path is nonunique and is not guaranteed to contain the oracle estimator; (2) even if a solution path is known to contain the oracle estimator, the optimal tuning parameter depends on many unknown factors and is hard to estimate. To address these two challenging issues, we first prove that an easy-to-calculate calibrated CCCP algorithm produces a consistent solution path which contains the oracle estimator with probability approaching one. Furthermore, we propose a high-dimensional BIC criterion and show that it can be applied to the solution path to select the optimal tuning parameter which asymptotically identifies the oracle estimator. The theory for a general class of nonconvex penalties in the ultra-high dimensional setup is established when the random errors follow the sub-Gaussian distribution. Monte Carlo studies confirm that the calibrated CCCP algorithm combined with the proposed high-dimensional BIC has desirable performance in identifying the underlying sparsity pattern for high-dimensional data analysis.

Published in at http://dx.doi.org/10.1214/13-AOS1159 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (9)

• Nearly unbiased variable selection under minimax concave penalty
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• High-dimensional generalized linear models and the lasso
• Calibrating nonconvex penalized regression in ultra-high dimension
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Adaptive Lasso for High Dimensional Regression and Gaussian Graphical Modeling
• Thresholded Lasso for high dimensional variable selection and statistical estimation

Cited by in corpus (9)

• Calibrating nonconvex penalized regression in ultra-high dimension
• Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
• Least Squares Approximation for a Distributed System
• Global solutions to folded concave penalized nonconvex learning
• Ultrahigh-dimensional Robust and Efficient Sparse Regression using Non-Concave Penalized Density Power Divergence
• Penalized Sparse Covariance Regression with High Dimensional Covariates
• Nonconvex penalized multitask regression using data depth-based penalties
• High-Dimensional Expected Shortfall Regression
• Accelerated Gradient Methods for Sparse Statistical Learning with Nonconvex Penalties