Hybrid safe-strong rules for efficient optimization in lasso-type problems
arXiv:1704.08742 · doi:10.1016/j.csda.2020.107063
Abstract
The lasso model has been widely used for model selection in data mining, machine learning, and high-dimensional statistical analysis. However, with the ultrahigh-dimensional, large-scale data sets now collected in many real-world applications, it is important to develop algorithms to solve the lasso that efficiently scale up to problems of this size. Discarding features from certain steps of the algorithm is a powerful technique for increasing efficiency and addressing the Big Data challenge. In this paper, we propose a family of hybrid safe-strong rules (HSSR) which incorporate safe screening rules into the sequential strong rule (SSR) to remove unnecessary computational burden. In particular, we present two instances of HSSR, namely SSR-Dome and SSR-BEDPP, for the standard lasso problem. We further extend SSR-BEDPP to the elastic net and group lasso problems to demonstrate the generalizability of the hybrid screening idea. Extensive numerical experiments with synthetic and real data sets are conducted for both the standard lasso and the group lasso problems. Results show that our proposed hybrid rules can substantially outperform existing state-of-the-art rules.
arXiv comment31 pages, 4 figures
References in corpus (17)
- Rejoinder to "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Pathwise coordinate optimization
- Coordinate descent algorithms for lasso penalized regression
- Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors
- Safe Feature Elimination for the LASSO and Sparse Supervised Learning Problems
- Screening Tests for Lasso Problems
- The biglasso Package: A Memory- and Computation-Efficient Solver for Lasso Model Fitting with Big Data in R
- Strong rules for nonconvex penalties and their implications for efficient algorithms in high-dimensional regression