More Powerful and General Selective Inference for Stepwise Feature Selection using the Homotopy Continuation Approach
arXiv:2012.13545
Abstract
Conditional selective inference (SI) has been actively studied as a new statistical inference framework for data-driven hypotheses. The basic idea of conditional SI is to make inferences conditional on the selection event characterized by a set of linear and/or quadratic inequalities. Conditional SI has been mainly studied in the context of feature selection such as stepwise feature selection (SFS). The main limitation of the existing conditional SI methods is the loss of power due to over-conditioning, which is required for computational tractability. In this study, we develop a more powerful and general conditional SI method for SFS using the homotopy method which enables us to overcome this limitation. The homotopy-based SI is especially effective for more complicated feature selection algorithms. As an example, we develop a conditional SI method for forward-backward SFS with AIC-based stopping criteria and show that it is not adversely affected by the increased complexity of the algorithm. We conduct several experiments to demonstrate the effectiveness and efficiency of the proposed method.
References in corpus (23)
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Valid post-selection inference
- A significance test for the lasso
- Piecewise linear regularized solution paths
- Optimal Inference After Model Selection
- Selective Sequential Model Selection
- Confidence Sets Based on Penalized Maximum Likelihood Estimators in Gaussian Regression
- A significance test for forward stepwise model selection
- Selective Inference for Group-Sparse Linear Models
- More powerful post-selection inference, with application to the Lasso
- Selective inference in regression models with groups of variables
- Selective inference after cross-validation
- Computing Valid p-value for Optimal Changepoint by Selective Inference using Dynamic Programming
- Parametric Programming Approach for More Powerful and General Lasso Selective Inference