An adaptive Ridge procedure for L0 regularization

Abstract

Penalized selection criteria like AIC or BIC are among the most popular methods for variable selection. Their theoretical properties have been studied intensively and are well understood, but making use of them in case of high-dimensional data is difficult due to the non-convex optimization problem induced by L0 penalties. An elegant solution to this problem is provided by the multi-step adaptive lasso, where iteratively weighted lasso problems are solved, whose weights are updated in such a way that the procedure converges towards selection with L0 penalties. In this paper we introduce an adaptive ridge procedure (AR) which mimics the adaptive lasso, but is based on weighted Ridge problems. After introducing AR its theoretical properties are studied in the particular case of orthogonal linear regression. For the non-orthogonal case extensive simulations are performed to assess the performance of AR. In case of Poisson regression and logistic regression it is illustrated how the iterative procedure of AR can be combined with iterative maximization procedures. The paper ends with an efficient implementation of AR in the context of least-squares segmentation.

References in corpus (7)

• Nearly unbiased variable selection under minimax concave penalty
• One-step sparse estimates in nonconcave penalized likelihood models
• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Analyzing genome-wide association studies with an FDR controlling modification of the Bayesian information criterion
• Asymptotic Bayes optimality under sparsity for generally distributed effect sizes under the alternative
• A model selection approach to genome wide association studies

Cited by in corpus (8)

• Scalable Sparse Cox's Regression for Large-Scale Survival Data via Broken Adaptive Ridge
• Spline Regression with Automatic Knot Selection
• L0 regularisation for the estimation of piecewise constant hazard rates in survival analysis
• Regression modelling of interval censored data based on the adaptive ridge procedure
• Regularized Bidimensional Estimation of the Hazard Rate
• Bayesian Error-in-Variables Models for the Identification of Power Networks
• Adaptive Ridge Approach to Heteroscedastic Regression
• Fixed effects selection in the linear mixed-effects model using adaptive ridge procedure for L0 penalty performance