Exact Post-Selection Inference for Sequential Regression Procedures
arXiv:1401.3889
Abstract
We propose new inference tools for forward stepwise regression, least angle regression, and the lasso. Assuming a Gaussian model for the observation vector y, we first describe a general scheme to perform valid inference after any selection event that can be characterized as y falling into a polyhedral set. This framework allows us to derive conditional (post-selection) hypothesis tests at any step of forward stepwise or least angle regression, or any step along the lasso regularization path, because, as it turns out, selection events for these procedures can be expressed as polyhedral constraints on y. The p-values associated with these tests are exactly uniform under the null distribution, in finite samples, yielding exact type I error control. The tests can also be inverted to produce confidence intervals for appropriate underlying regression parameters. The R package "selectiveInference", freely available on the CRAN repository, implements the new inference tools described in this paper.
arXiv comment26 pages, 5 figures
References in corpus (16)
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
- Valid post-selection inference
- A significance test for the lasso
- Optimal Inference After Model Selection
- Exact Post Model Selection Inference for Marginal Screening
- Tests in adaptive regression via the Kac-Rice formula
Cited by in corpus (33)
- Optimal Inference After Model Selection
- Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach
- High-dimensional inference in misspecified linear models
- Selective Sequential Model Selection
- High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normality
- Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
- How much does your data exploration overfit? Controlling bias via information usage
- A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models
- Inference in High Dimensions with the Penalized Score Test
- Selective inference with a randomized response
- Local and Global Inference for High Dimensional Nonparanormal Graphical Models
- A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
- On Online Control of False Discovery Rate
- Selective inference with unknown variance via the square-root LASSO
- Concise Summarization of Heterogeneous Treatment Effect Using Total Variation Regularized Regression
- Sequential Selection Procedures and False Discovery Rate Control
- Floodgate: inference for model-free variable importance
- A Minimax Theory for Adaptive Data Analysis
- A Likelihood Ratio Framework for High Dimensional Semiparametric Regression
- Constrained High Dimensional Statistical Inference
- Accumulation tests for FDR control in ordered hypothesis testing
- Tractable Post-Selection Maximum Likelihood Inference for the Lasso
- Asymptotics of selective inference
- Inference After Selecting Plausibly Valid Instruments with Application to Mendelian Randomization
- EigenPrism: Inference for High-Dimensional Signal-to-Noise Ratios
- Uncertainty Quantification Under Group Sparsity
- Tuning-free ridge estimators for high-dimensional generalized linear models
- Selective Inference and Learning Mixed Graphical Models
- Construction Cost Index Forecasting: A Multi-feature Fusion Approach
- Inject Machine Learning into Significance Test for Misspecified Linear Models
- Efficient Test-based Variable Selection for High-dimensional Linear Models
- gLOP: the global and Local Penalty for Capturing Predictive Heterogeneity
- DVS: Deep Visibility Series and its Application in Construction Cost Index Forecasting