A significance test for forward stepwise model selection

Abstract

We apply the methods developed by Lockhart et al. (2013) and Taylor et al. (2013) on significance tests for penalized regression to forward stepwise model selection. A general framework for selection procedures described by quadratic inequalities includes a variant of forward stepwise with grouped variables, allowing us to handle categorical variables and factor models. We provide an algorithm to compute a new statistic with an exact null distribution conditional on the outcome of the model selection procedure. This new statistic, which we denote $T\chi$, has a truncated $\chi$ distribution under the global null. We apply this test in forward stepwise iteratively on the residual after each step. The resulting method has the computational strengths of stepwise selection and addresses the problem of invalid test statistics due to model selection. We illustrate the flexibility of this method by applying it to several specialized applications of forward stepwise including a hierarchical interactions model and a recently described additive model that adaptively chooses between linear and nonlinear effects for each variable.

References in corpus (1)

• A significance test for the lasso

Cited by in corpus (15)

• Optimal Inference After Model Selection
• Selective Sequential Model Selection
• Selective inference for effect modification via the lasso
• Selective inference in regression models with groups of variables
• Selective sampling after solving a convex problem
• More Powerful Conditional Selective Inference for Generalized Lasso by Parametric Programming
• Computing Valid p-value for Optimal Changepoint by Selective Inference using Dynamic Programming
• Sequential Selection Procedures and False Discovery Rate Control
• Parametric Programming Approach for More Powerful and General Lasso Selective Inference
• Quantifying Statistical Significance of Neural Network-based Image Segmentation by Selective Inference
• Valid Post-Detection Inference for Change Points Identified Using Trend Filtering
• Valid and Exact Statistical Inference for Multi-dimensional Multiple Change-Points by Selective Inference
• More Powerful and General Selective Inference for Stepwise Feature Selection using the Homotopy Continuation Approach
• A Selective Approach to Internal Inference
• Bayesian Selective Inference: Non-informative Priors