A lasso for hierarchical interactions
arXiv:1205.5050 · doi:10.1214/13-AOS1096
Abstract
We add a set of convex constraints to the lasso to produce sparse interaction models that honor the hierarchy restriction that an interaction only be included in a model if one or both variables are marginally important. We give a precise characterization of the effect of this hierarchy constraint, prove that hierarchy holds with probability one and derive an unbiased estimate for the degrees of freedom of our estimator. A bound on this estimate reveals the amount of fitting "saved" by the hierarchy constraint. We distinguish between parameter sparsity - the number of nonzero coefficients - and practical sparsity - the number of raw variables one must measure to make a new prediction. Hierarchy focuses on the latter, which is more closely tied to important data collection concerns such as cost, time and effort. We develop an algorithm, available in the R package hierNet, and perform an empirical study of our method.
arXiv commentPublished in at http://dx.doi.org/10.1214/13-AOS1096 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
References in corpus (18)
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- On the "degrees of freedom" of the lasso
- The composite absolute penalties family for grouped and hierarchical variable selection
- The solution path of the generalized lasso
- Structured Variable Selection with Sparsity-Inducing Norms
- Degrees of freedom in lasso problems
- Group Lasso with Overlaps: the Latent Group Lasso approach
- Structured variable selection and estimation
- The log-linear group-lasso estimator and its asymptotic properties
Cited by in corpus (23)
- Supersparse Linear Integer Models for Optimized Medical Scoring Systems
- Hierarchical Sparse Modeling: A Choice of Two Group Lasso Formulations
- Convex Modeling of Interactions with Strong Heredity
- Variable selection for general index models via sliced inverse regression
- An Ordered Lasso and Sparse Time-Lagged Regression
- Iteratively reweighted adaptive lasso for conditional heteroscedastic time series with applications to AR-ARCH type processes
- Prediction of hierarchical time series using structured regularization and its application to artificial neural networks
- On the Sensitivity of the Lasso to the Number of Predictor Variables
- Semiparametric integrative interaction analysis for non-small-cell lung cancer
- Convex hierarchical testing of interactions
- A Hierarchical Integrative Group LASSO (HiGLASSO) Framework for Analyzing Environmental Mixtures
- mpower: An R Package for Power Analysis of Exposure Mixture Studies via Monte Carlo Simulations
- Using Machine Learning to Test Causal Hypotheses in Conjoint Analysis
- An analysis of penalized interaction models
- Parameter Estimation with the Ordered Regularization via an Alternating Direction Method of Multipliers
- A scalable hierarchical lasso for gene-environment interactions
- An ADMM approach for multi-response regression with overlapping groups and interaction effects
- Approximate Laplace approximations for scalable model selection
- Adaptive Bi-Level Variable Selection of Conditional Main Effects for Generalized Linear Models
- IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms
- Predicting Census Survey Response Rates With Parsimonious Additive Models and Structured Interactions
- Interaction as Interference: A Quantum-Inspired Aggregation Approach
- Topological Techniques in Model Selection