Nonconvex Sparse Logistic Regression with Weakly Convex Regularization

Abstract

In this work we propose to fit a sparse logistic regression model by a weakly convex regularized nonconvex optimization problem. The idea is based on the finding that a weakly convex function as an approximation of the ${\ell }_0$ pseudo norm is able to better induce sparsity than the commonly used ${\ell }_1$ norm. For a class of weakly convex sparsity inducing functions, we prove the nonconvexity of the corresponding sparse logistic regression problem, and study its local optimality conditions and the choice of the regularization parameter to exclude trivial solutions. Despite the nonconvexity, a method based on proximal gradient descent is used to solve the general weakly convex sparse logistic regression, and its convergence behavior is studied theoretically. Then the general framework is applied to a specific weakly convex function, and a necessary and sufficient local optimality condition is provided. The solution method is instantiated in this case as an iterative firm-shrinkage algorithm, and its effectiveness is demonstrated in numerical experiments by both randomly generated and real datasets.

References in corpus (5)

• Nearly unbiased variable selection under minimax concave penalty
• CVXPY: A Python-Embedded Modeling Language for Convex Optimization
• Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima
• Convergence Analysis of Proximal Gradient with Momentum for Nonconvex Optimization
• Robustness of Sparse Recovery via $F$-minimization: A Topological Viewpoint

Cited by in corpus (7)

• Deep Learning Methods for Solving Linear Inverse Problems: Research Directions and Paradigms
• Linearly-involved Moreau-Enhanced-over-Subspace Model: Debiased Sparse Modeling and Stable Outlier-Robust Regression
• Variable Smoothing for Weakly Convex Composite Functions
• Monotone Lipschitz-Gradient Denoiser: Explainability of Operator Regularization Approaches Free From Lipschitz Constant Control
• Understanding Notions of Stationarity in Non-Smooth Optimization
• On the Energy Self-Sustainability of IoT via Distributed Compressed Sensing
• Run-and-Inspect Method for Nonconvex Optimization and Global Optimality Bounds for R-Local Minimizers