Reluctant Interaction Modeling

Abstract

Including pairwise interactions between the predictors of a regression model can produce better predicting models. However, to fit such interaction models on typical data sets in biology and other fields can often require solving enormous variable selection problems with billions of interactions. The scale of such problems demands methods that are computationally cheap (both in time and memory) yet still have sound statistical properties. Motivated by these large-scale problem sizes, we adopt a very simple guiding principle: One should prefer main effects over interactions if all else is equal. This "reluctance" to interactions, while reminiscent of the hierarchy principle for interactions, is much less restrictive. We design a computationally efficient method built upon this principle and provide theoretical results indicating favorable statistical properties. Empirical results show dramatic computational improvement without sacrificing statistical properties. For example, the proposed method can solve a problem with 10 billion interactions with 5-fold cross-validation in under 7 hours on a single CPU.

References in corpus (7)

• The composite absolute penalties family for grouped and hierarchical variable selection
• Structured variable selection and estimation
• Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression
• Rare Feature Selection in High Dimensions
• The Simulator: An Engine to Streamline Simulations
• Penalized Interaction Estimation for Ultrahigh Dimensional Quadratic Regression
• Strong Sure Screening of Ultra-high Dimensional Data with Interaction Effects

Cited by in corpus (6)

• Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression
• Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Recovering Identifiable Additive Models
• A reluctant additive model framework for interpretable nonlinear individualized treatment rules
• Reluctant generalized additive modeling
• Learning High Order Feature Interactions with Fine Control Kernels
• Online Interaction Detection for Click-Through Rate Prediction