Variable selection in nonparametric additive models

Abstract

We consider a nonparametric additive model of a conditional mean function in which the number of variables and additive components may be larger than the sample size but the number of nonzero additive components is "small" relative to the sample size. The statistical problem is to determine which additive components are nonzero. The additive components are approximated by truncated series expansions with B-spline bases. With this approximation, the problem of component selection becomes that of selecting the groups of coefficients in the expansion. We apply the adaptive group Lasso to select nonzero components, using the group Lasso to obtain an initial estimator and reduce the dimension of the problem. We give conditions under which the group Lasso selects a model whose number of components is comparable with the underlying model, and the adaptive group Lasso selects the nonzero components correctly with probability approaching one as the sample size increases and achieves the optimal rate of convergence. The results of Monte Carlo experiments show that the adaptive group Lasso procedure works well with samples of moderate size. A data example is used to illustrate the application of the proposed method.

Published in at http://dx.doi.org/10.1214/09-AOS781 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (9)

• Nearly unbiased variable selection under minimax concave penalty
• Consistency of the group Lasso and multiple kernel learning
• Component selection and smoothing in multivariate nonparametric regression
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• High-dimensional generalized linear models and the lasso
• Sparsity oracle inequalities for the Lasso
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models

Cited by in corpus (5)

• Group Lasso for high dimensional sparse quantile regression models
• Flexible Shrinkage Estimation in High-Dimensional Varying Coefficient Models
• Grouped Variable Selection via Nested Spike and Slab Priors
• LASSO ISOtone for High Dimensional Additive Isotonic Regression
• Semiparametric Bayesian Information Criterion for Model Selection in Ultra-high Dimensional Additive Models