Structured variable selection and estimation

Abstract

In linear regression problems with related predictors, it is desirable to do variable selection and estimation by maintaining the hierarchical or structural relationships among predictors. In this paper we propose non-negative garrote methods that can naturally incorporate such relationships defined through effect heredity principles or marginality principles. We show that the methods are very easy to compute and enjoy nice theoretical properties. We also show that the methods can be easily extended to deal with more general regression problems such as generalized linear models. Simulations and real examples are used to illustrate the merits of the proposed methods.

Published in at http://dx.doi.org/10.1214/09-AOAS254 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (11)

• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• The composite absolute penalties family for grouped and hierarchical variable selection

Cited by in corpus (24)

• A lasso for hierarchical interactions
• STAR: A general interactive framework for FDR control under structural constraints
• Hierarchical Sparse Modeling: A Choice of Two Group Lasso Formulations
• Multicategory vertex discriminant analysis for high-dimensional data
• The log-linear group-lasso estimator and its asymptotic properties
• Interaction Pursuit with Feature Screening and Selection
• GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions
• Reluctant Interaction Modeling
• Structured variable selection in support vector machines
• Regularizers for Structured Sparsity
• An analysis of penalized interaction models
• Interaction pursuit in high-dimensional multi-response regression via distance correlation
• Penalized Interaction Estimation for Ultrahigh Dimensional Quadratic Regression
• Model Selection for High Dimensional Quadratic Regression via Regularization
• Modelling Interactions in High-dimensional Data with Backtracking
• Bayesian projection approaches to variable selection and exploring model uncertainty
• Strong Sure Screening of Ultra-high Dimensional Data with Interaction Effects
• A Note on High Dimensional Linear Regression with Interactions
• Hierarchical sparsity priors for regression models
• svReg: Structural Varying-coefficient regression to differentiate how regional brain atrophy affects motor impairment for Huntington disease severity groups
• Bayesian Variable Selection on Model Spaces Constrained by Heredity Conditions
• An Easy-to-Implement Hierarchical Standardization for Variable Selection Under Strong Heredity Constraint
• A fresh look at effect aliasing and interactions: some new wine in old bottles
• A-ComVar: A Flexible Extension of Common Variance Design