Sparse Additive Models

Abstract

We present a new class of methods for high-dimensional nonparametric regression and classification called sparse additive models (SpAM). Our methods combine ideas from sparse linear modeling and additive nonparametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. SpAM is closely related to the COSSO model of Lin and Zhang (2006), but decouples smoothing and sparsity, enabling the use of arbitrary nonparametric smoothers. An analysis of the theoretical properties of SpAM is given. We also study a greedy estimator that is a nonparametric version of forward stepwise regression. Empirical results on synthetic and real data are presented, showing that SpAM can be effective in fitting sparse nonparametric models in high dimensional data.

References in corpus (2)

• Component selection and smoothing in multivariate nonparametric regression
• Sharp thresholds for high-dimensional and noisy recovery of sparsity

Cited by in corpus (6)

• Support union recovery in high-dimensional multivariate regression
• Sure Independence Screening for Ultra-High Dimensional Feature Space
• Compositional ADAM: An Adaptive Compositional Solver
• Simultaneous support recovery in high dimensions: Benefits and perils of block ${\ell }_1/{\ell }_{\infty }$-regularization
• Reluctant generalized additive modeling
• Autoregressive Process Modeling via the Lasso Procedure