Simultaneous Safe Screening of Features and Samples in Doubly Sparse Modeling
arXiv:1602.02485
Abstract
The problem of learning a sparse model is conceptually interpreted as the process of identifying active features/samples and then optimizing the model over them. Recently introduced safe screening allows us to identify a part of non-active features/samples. So far, safe screening has been individually studied either for feature screening or for sample screening. In this paper, we introduce a new approach for safely screening features and samples simultaneously by alternatively iterating feature and sample screening steps. A significant advantage of considering them simultaneously rather than individually is that they have a synergy effect in the sense that the results of the previous safe feature screening can be exploited for improving the next safe sample screening performances, and vice-versa. We first theoretically investigate the synergy effect, and then illustrate the practical advantage through intensive numerical experiments for problems with large numbers of features and samples.
References in corpus (8)
- Lasso Screening Rules via Dual Polytope Projection
- Safe Feature Elimination for the LASSO and Sparse Supervised Learning Problems
- Screening Tests for Lasso Problems
- Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization
- Safe Screening With Variational Inequalities and Its Application to LASSO
- Scaling SVM and Least Absolute Deviations via Exact Data Reduction
- A Safe Screening Rule for Sparse Logistic Regression
- GAP Safe screening rules for sparse multi-task and multi-class models