A Safe Screening Rule for Sparse Logistic Regression
arXiv:1307.4145
Abstract
The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper, we present a fast and effective sparse logistic regression screening rule (Slores) to identify the 0 components in the solution vector, which may lead to a substantial reduction in the number of features to be entered to the optimization. An appealing feature of Slores is that the data set needs to be scanned only once to run the screening and its computational cost is negligible compared to that of solving the sparse logistic regression problem. Moreover, Slores is independent of solvers for sparse logistic regression, thus Slores can be integrated with any existing solver to improve the efficiency. We have evaluated Slores using high-dimensional data sets from different applications. Extensive experimental results demonstrate that Slores outperforms the existing state-of-the-art screening rules and the efficiency of solving sparse logistic regression is improved by one magnitude in general.
References in corpus (1)
Cited by in corpus (20)
- Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction
- GAP Safe screening rules for sparse multi-task and multi-class models
- Simultaneous Safe Screening of Features and Samples in Doubly Sparse Modeling
- Gap Safe screening rules for sparsity enforcing penalties
- Safe Screening for Multi-Task Feature Learning with Multiple Data Matrices
- Secure Approximation Guarantee for Cryptographically Private Empirical Risk Minimization
- Safe Screening for the Generalized Conditional Gradient Method
- Regularization Path of Cross-Validation Error Lower Bounds
- Screening Rules and its Complexity for Active Set Identification
- One to beat them all: "RYU" -- a unifying framework for the construction of safe balls
- Safe Feature Pruning for Sparse High-Order Interaction Models
- Safe Active Feature Selection for Sparse Learning
- Interval-based Prediction Uncertainty Bound Computation in Learning with Missing Values
- Safe Sample Screening for Robust Support Vector Machine
- Quick sensitivity analysis for incremental data modification and its application to leave-one-out CV in linear classification problems
- On Inductive Biases for Machine Learning in Data Constrained Settings
- Safe Screening for Sparse Conditional Random Fields
- Screening for Sparse Online Learning
- Tighter Bound Estimation of Sensitivity Analysis for Incremental and Decremental Data Modification
- On Newton Screening