Exact Post-Selection Inference for Sequential Regression Procedures

Abstract

We propose new inference tools for forward stepwise regression, least angle regression, and the lasso. Assuming a Gaussian model for the observation vector y, we first describe a general scheme to perform valid inference after any selection event that can be characterized as y falling into a polyhedral set. This framework allows us to derive conditional (post-selection) hypothesis tests at any step of forward stepwise or least angle regression, or any step along the lasso regularization path, because, as it turns out, selection events for these procedures can be expressed as polyhedral constraints on y. The p-values associated with these tests are exactly uniform under the null distribution, in finite samples, yielding exact type I error control. The tests can also be inverted to produce confidence intervals for appropriate underlying regression parameters. The R package "selectiveInference", freely available on the CRAN repository, implements the new inference tools described in this paper.

26 pages, 5 figures

References in corpus (2)

• Optimal Inference After Model Selection
• Exact Post Model Selection Inference for Marginal Screening

Cited by in corpus (32)

• Optimal Inference After Model Selection
• Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach
• High-dimensional inference in misspecified linear models
• Selective Sequential Model Selection
• High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normality
• Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
• How much does your data exploration overfit? Controlling bias via information usage
• A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models
• Inference in High Dimensions with the Penalized Score Test
• Selective inference with a randomized response
• Local and Global Inference for High Dimensional Nonparanormal Graphical Models
• A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
• On Online Control of False Discovery Rate
• Selective inference with unknown variance via the square-root LASSO
• Concise Summarization of Heterogeneous Treatment Effect Using Total Variation Regularized Regression
• Floodgate: inference for model-free variable importance
• A Minimax Theory for Adaptive Data Analysis
• A Likelihood Ratio Framework for High Dimensional Semiparametric Regression
• Constrained High Dimensional Statistical Inference
• Tractable Post-Selection Maximum Likelihood Inference for the Lasso
• Asymptotics of selective inference
• Accumulation tests for FDR control in ordered hypothesis testing
• EigenPrism: Inference for High-Dimensional Signal-to-Noise Ratios
• Inference After Selecting Plausibly Valid Instruments with Application to Mendelian Randomization
• Uncertainty Quantification Under Group Sparsity
• Selective Inference and Learning Mixed Graphical Models
• Construction Cost Index Forecasting: A Multi-feature Fusion Approach
• Tuning-free ridge estimators for high-dimensional generalized linear models
• gLOP: the global and Local Penalty for Capturing Predictive Heterogeneity
• Efficient Test-based Variable Selection for High-dimensional Linear Models
• Inject Machine Learning into Significance Test for Misspecified Linear Models
• DVS: Deep Visibility Series and its Application in Construction Cost Index Forecasting