A Selective Review of Group Selection in High-Dimensional Models

Abstract

Grouping structures arise naturally in many statistical modeling problems. Several methods have been proposed for variable selection that respect grouping structure in variables. Examples include the group LASSO and several concave group selection methods. In this article, we give a selective review of group selection concerning methodological developments, theoretical properties and computational algorithms. We pay particular attention to group selection methods involving concave penalties. We address both group selection and bi-level selection methods. We describe several applications of these methods in nonparametric additive models, semiparametric regression, seemingly unrelated regressions, genomic data analysis and genome wide association studies. We also highlight some issues that require further study.

arXiv commentPublished in at http://dx.doi.org/10.1214/12-STS392 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (30)

• Least Angle Regression
• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Nearly unbiased variable selection under minimax concave penalty
• High-dimensional graphs and variable selection with the Lasso
• Simultaneous analysis of Lasso and Dantzig selector
• Pathwise coordinate optimization
• Nonconcave penalized likelihood with a diverging number of parameters
• On the conditions used to prove oracle results for the Lasso
• Consistency of the group Lasso and multiple kernel learning
• One-step sparse estimates in nonconcave penalized likelihood models
• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• The composite absolute penalties family for grouped and hierarchical variable selection
• Component selection and smoothing in multivariate nonparametric regression
• Variable selection in nonparametric additive models
• Coordinate descent algorithms for lasso penalized regression
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• High-dimensional additive modeling
• Support union recovery in high-dimensional multivariate regression
• The Dantzig selector and sparsity oracle inequalities
• Some sharp performance bounds for least squares regression with $L_1$ regularization
• Consistent group selection in high-dimensional linear regression

Cited by in corpus (20)

• Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors
• Partially linear additive quantile regression in ultra-high dimension
• Bayesian Variable Selection and Estimation for Group Lasso
• Group Sparse Recovery via the ${\ell }^0({\ell }^2)$ Penalty: Theory and Algorithm
• The iterative reweighted Mixed-Norm Estimate for spatio-temporal MEG/EEG source reconstruction
• Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models
• Vector-Valued Graph Trend Filtering with Non-Convex Penalties
• Simultaneous sparse estimation of canonical vectors in the p>>N setting
• Sparse quadratic classification rules via linear dimension reduction
• Robust Finite Mixture Regression for Heterogeneous Targets
• Adaptive group-regularized logistic elastic net regression
• Sparse Model Identification and Learning for Ultra-high-dimensional Additive Partially Linear Models
• Scalable Bayesian regression in high dimensions with multiple data sources
• A High-dimensional M-estimator Framework for Bi-level Variable Selection
• A Convex-Nonconvex Strategy for Grouped Variable Selection
• Partitioned Least Squares
• Introduction to the Special Issue on Sparsity and Regularization Methods
• Simultaneous nonparametric regression in RADWT dictionaries
• Group Spike and Slab Variational Bayes
• A Convex-Nonconvex Framework for Enhancing Minimization Induced Penalties