Dynamic Sasvi: Strong Safe Screening for Norm-Regularized Least Squares
arXiv:2102.04108
Abstract
A recently introduced technique for a sparse optimization problem called "safe screening" allows us to identify irrelevant variables in the early stage of optimization. In this paper, we first propose a flexible framework for safe screening based on the Fenchel-Rockafellar duality and then derive a strong safe screening rule for norm-regularized least squares by the framework. We call the proposed screening rule for norm-regularized least squares "dynamic Sasvi" because it can be interpreted as a generalization of Sasvi. Unlike the original Sasvi, it does not require the exact solution of a more strongly regularized problem; hence, it works safely in practice. We show that our screening rule can eliminate more features and increase the speed of the solver in comparison with other screening rules both theoretically and experimentally.
References in corpus (7)
- Nearly unbiased variable selection under minimax concave penalty
- Pathwise coordinate optimization
- Safe Feature Elimination for the LASSO and Sparse Supervised Learning Problems
- Screening Tests for Lasso Problems
- Dynamic Screening: Accelerating First-Order Algorithms for the Lasso and Group-Lasso
- Safe Screening With Variational Inequalities and Its Application to LASSO
- GAP Safe screening rules for sparse multi-task and multi-class models