Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM

Abstract

Feature selection in learning to rank has recently emerged as a crucial issue. Whereas several preprocessing approaches have been proposed, only a few works have been focused on integrating the feature selection into the learning process. In this work, we propose a general framework for feature selection in learning to rank using SVM with a sparse regularization term. We investigate both classical convex regularizations such as $\ell \_1$ or weighted $\ell \_1$ and non-convex regularization terms such as log penalty, Minimax Concave Penalty (MCP) or $\ell \_p$ pseudo norm with $p\textless{}1$. Two algorithms are proposed, first an accelerated proximal approach for solving the convex problems, second a reweighted $\ell \_1$ scheme to address the non-convex regularizations. We conduct intensive experiments on nine datasets from Letor 3.0 and Letor 4.0 corpora. Numerical results show that the use of non-convex regularizations we propose leads to more sparsity in the resulting models while prediction performance is preserved. The number of features is decreased by up to a factor of six compared to the $\ell \_1$ regularization. In addition, the software is publicly available on the web.

References in corpus (1)

• Nearly unbiased variable selection under minimax concave penalty

Cited by in corpus (10)

• Non-convex regularization in remote sensing
• A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression
• Non-convex Lasso-kind approach to compressed sensing for finite-valued signals
• Importance sampling strategy for non-convex randomized block-coordinate descent
• A review on ranking problems in statistical learning
• An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines
• Stochastic Variance Reduction Gradient for a Non-convex Problem Using Graduated Optimization
• Sparse hierarchical interaction learning with epigraphical projection
• Oracle inequalities for ranking and U-processes with Lasso penalty
• Greedy methods, randomization approaches and multi-arm bandit algorithms for efficient sparsity-constrained optimization