Tractable Post-Selection Maximum Likelihood Inference for the Lasso
arXiv:1705.09417
Abstract
Applying standard statistical methods after model selection may yield inefficient estimators and hypothesis tests that fail to achieve nominal type-I error rates. The main issue is the fact that the post-selection distribution of the data differs from the original distribution. In particular, the observed data is constrained to lie in a subset of the original sample space that is determined by the selected model. This often makes the post-selection likelihood of the observed data intractable and maximum likelihood inference difficult. In this work, we get around the intractable likelihood by generating noisy unbiased estimates of the post-selection score function and using them in a stochastic ascent algorithm that yields correct post-selection maximum likelihood estimates. We apply the proposed technique to the problem of estimating linear models selected by the lasso. In an asymptotic analysis the resulting estimates are shown to be consistent for the selected parameters and to have a limiting truncated normal distribution. Confidence intervals constructed based on the asymptotic distribution obtain close to nominal coverage rates in all simulation settings considered, and the point estimates are shown to be superior to the lasso estimates when the true model is sparse.
References in corpus (9)
- Valid post-selection inference
- A significance test for the lasso
- Optimal Inference After Model Selection
- Exact Post Model Selection Inference for Marginal Screening
- Exact Post-Selection Inference for Sequential Regression Procedures
- Estimation after Parameter Selection: Performance Analysis and Estimation Methods
- Selective inference with a randomized response
- Trust, but verify: benefits and pitfalls of least-squares refitting in high dimensions
- Selective Correlations - the conditional estimators