Interaction Pursuit with Feature Screening and Selection

Abstract

Understanding how features interact with each other is of paramount importance in many scientific discoveries and contemporary applications. Yet interaction identification becomes challenging even for a moderate number of covariates. In this paper, we suggest an efficient and flexible procedure, called the interaction pursuit (IP), for interaction identification in ultra-high dimensions. The suggested method first reduces the number of interactions and main effects to a moderate scale by a new feature screening approach, and then selects important interactions and main effects in the reduced feature space using regularization methods. Compared to existing approaches, our method screens interactions separately from main effects and thus can be more effective in interaction screening. Under a fairly general framework, we establish that for both interactions and main effects, the method enjoys the sure screening property in screening and oracle inequalities in selection. Our method and theoretical results are supported by several simulation and real data examples.

arXiv comment34 pages for the main text including 7 figures, 53 pages for the Supplementary Material

References in corpus (8)

• Nearly unbiased variable selection under minimax concave penalty
• Simultaneous analysis of Lasso and Dantzig selector
• Sure independence screening in generalized linear models with NP-dimensionality
• Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data
• Structured variable selection and estimation
• High-dimensional variable selection via tilting
• Asymptotic properties for combined $L_1$ and concave regularization
• Model Selection for High Dimensional Quadratic Regression via Regularization

Cited by in corpus (9)

• Detecting Statistical Interactions from Neural Network Weights
• RaSE: A Variable Screening Framework via Random Subspace Ensembles
• Reluctant Interaction Modeling
• The Kendall Interaction Filter for Variable Interaction Screening in Ultra High Dimensional Classification Problems
• Interaction pursuit in high-dimensional multi-response regression via distance correlation
• RANK: Large-Scale Inference with Graphical Nonlinear Knockoffs
• Covariate Assisted Variable Ranking
• High-dimensional Adaptive Minimax Sparse Estimation with Interactions
• Controlling the False Discovery Rate for Binary Feature Selection via Knockoff