Uniformly Valid Confidence Sets Based on the Lasso

Abstract

In a linear regression model of fixed dimension $p\le n$, we construct confidence regions for the unknown parameter vector based on the Lasso estimator that uniformly and exactly hold the prescribed in finite samples as well as in an asymptotic setup. We thereby quantify estimation uncertainty as well as the "post-model selection error" of this estimator. More concretely, in finite samples with Gaussian errors and asymptotically in the case where the Lasso estimator is tuned to perform conservative model selection, we derive exact formulas for computing the minimal coverage probability over the entire parameter space for a large class of shapes for the confidence sets, thus enabling the construction of valid confidence regions based on the Lasso estimator in these settings. The choice of shape for the confidence sets and comparison with the confidence ellipse based on the least-squares estimator is also discussed. Moreover, in the case where the Lasso estimator is tuned to enable consistent model selection, we give a simple confidence region with minimal coverage probability converging to one. Finally, we also treat the case of unknown error variance and present some ideas for extensions.

Some typos corrected, updated references

References in corpus (3)

• Piecewise linear regularized solution paths
• On the Distribution, Model Selection Properties and Uniqueness of the Lasso Estimator in Low and High Dimensions
• On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding

Cited by in corpus (4)

• On the Distribution, Model Selection Properties and Uniqueness of the Lasso Estimator in Low and High Dimensions
• Constructing confidence sets after lasso selection by randomized estimator augmentation
• Uniformly Valid Inference Based on the Lasso in Linear Mixed Models
• Honest confidence sets for high-dimensional regression by projection and shrinkage