Ultrahigh-dimensional Robust and Efficient Sparse Regression using Non-Concave Penalized Density Power Divergence
arXiv:1802.04906 · doi:10.1109/TIT.2020.3013015
Abstract
We propose a sparse regression method based on the non-concave penalized density power divergence loss function which is robust against infinitesimal contamination in very high dimensionality. Present methods of sparse and robust regression are based on -penalization, and their theoretical properties are not well-investigated. In contrast, we use a general class of folded concave penalties that ensure sparse recovery and consistent estimation of regression coefficients. We propose an alternating algorithm based on the Concave-Convex procedure to obtain our estimate, and demonstrate its robustness properties using influence function analysis. Under some conditions on the fixed design matrix and penalty function, we prove that this estimator possesses large-sample oracle properties in an ultrahigh-dimensional regime. The performance and effectiveness of our proposed method for parameter estimation and prediction compared to state-of-the-art are demonstrated through simulation studies.
arXiv commentAccepted in IEEE Transactions on Information Theory
References in corpus (12)
- Nearly unbiased variable selection under minimax concave penalty
- Simultaneous analysis of Lasso and Dantzig selector
- Nonconcave penalized likelihood with a diverging number of parameters
- One-step sparse estimates in nonconcave penalized likelihood models
- The sparsity and bias of the Lasso selection in high-dimensional linear regression
- Tuning parameter selection in high dimensional penalized likelihood
- A unified approach to model selection and sparse recovery using regularized least squares
- Sparse least trimmed squares regression for analyzing high-dimensional large data sets
- Calibrating nonconvex penalized regression in ultra-high dimension
- Robust Estimation in Generalised Linear Models : The Density Power Divergence Approach
- Robust and Sparse Regression via -divergence
- Nonconvex penalized multitask regression using data depth-based penalties