Nonconvex Penalization in Sparse Estimation: An Approach Based on the Bernstein Function

Abstract

In this paper we study nonconvex penalization using Bernstein functions whose first-order derivatives are completely monotone. The Bernstein function can induce a class of nonconvex penalty functions for high-dimensional sparse estimation problems. We derive a thresholding function based on the Bernstein penalty and discuss some important mathematical properties in sparsity modeling. We show that a coordinate descent algorithm is especially appropriate for regression problems penalized by the Bernstein function. We also consider the application of the Bernstein penalty in classification problems and devise a proximal alternating linearized minimization method. Based on theory of the Kurdyka-Lojasiewicz inequality, we conduct convergence analysis of these alternating iteration procedures. We particularly exemplify a family of Bernstein nonconvex penalties based on a generalized Gamma measure and conduct empirical analysis for this family.

arXiv admin note: substantial text overlap with arXiv:1312.4719

References in corpus (4)

• Nearly unbiased variable selection under minimax concave penalty
• A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems
• Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima
• Kinetic Energy Plus Penalty Functions for Sparse Estimation