MM Algorithms for Minimizing Nonsmoothly Penalized Objective Functions

Abstract

In this paper, we propose a general class of algorithms for optimizing an extensive variety of nonsmoothly penalized objective functions that satisfy certain regularity conditions. The proposed framework utilizes the majorization-minimization (MM) algorithm as its core optimization engine. The resulting algorithms rely on iterated soft-thresholding, implemented componentwise, allowing for fast, stable updating that avoids the need for any high-dimensional matrix inversion. We establish a local convergence theory for this class of algorithms under weaker assumptions than previously considered in the statistical literature. We also demonstrate the exceptional effectiveness of new acceleration methods, originally proposed for the EM algorithm, in this class of problems. Simulation results and a microarray data example are provided to demonstrate the algorithm's capabilities and versatility.

A revised version of this paper has been published in the Electronic Journal of Statistics

References in corpus (18)

• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Nearly unbiased variable selection under minimax concave penalty
• On the adaptive elastic-net with a diverging number of parameters
• One-step sparse estimates in nonconcave penalized likelihood models
• Coordinate descent algorithms for lasso penalized regression
• Piecewise linear regularized solution paths
• Variable selection using MM algorithms
• Ultrahigh dimensional variable selection: beyond the linear model
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models

Cited by in corpus (6)

• A Class of Nonconvex Penalties Preserving Overall Convexity in Optimization-Based Mean Filtering
• Robust Parametric Classification and Variable Selection by a Minimum Distance Criterion
• Sparsity-based Correction of Exponential Artifacts
• A hierarchical Bayesian perspective on majorization-minimization for non-convex sparse regression: application to M/EEG source imaging
• Preconditioned P-ULA for Joint Deconvolution-Segmentation of Ultrasound Images -- Extended Version
• MM for Penalized Estimation