Constructing confidence sets after lasso selection by randomized estimator augmentation
arXiv:1904.08018
Abstract
Although a few methods have been developed recently for building confidence intervals after model selection, how to construct confidence sets for joint post-selection inference is still an open question. In this paper, we develop a new method to construct confidence sets after lasso variable selection, with strong numerical support for its accuracy and effectiveness. A key component of our method is to sample from the conditional distribution of the response given the lasso active set, which, in general, is very challenging due to the tiny probability of the conditioning event. We overcome this technical difficulty by using estimator augmentation to simulate from this conditional distribution via Markov chain Monte Carlo given any estimate of the mean of . We then incorporate a randomization step for the estimate in our sampling procedure, which may be interpreted as simulating from a posterior predictive distribution by averaging over the uncertainty in . Our Monte Carlo samples offer great flexibility in the construction of confidence sets for multiple parameters. Extensive numerical results show that our method is able to construct confidence sets with the desired coverage rate and, moreover, that the diameter and volume of our confidence sets are substantially smaller in comparison with a state-of-the-art method.
arXiv comment31 pages, 7 figures, 4 tables
References in corpus (5)
- Valid post-selection inference
- More powerful post-selection inference, with application to the Lasso
- Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
- Uniformly Valid Confidence Sets Based on the Lasso
- On the Distribution, Model Selection Properties and Uniqueness of the Lasso Estimator in Low and High Dimensions