Confidence Sets Based on Penalized Maximum Likelihood Estimators in Gaussian Regression

Abstract

Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the maximum likelihood estimator. In the case where the penalized estimators are tuned to possess the `sparsity property', the intervals based on these estimators are larger than the standard interval by an order of magnitude. Furthermore, a simple asymptotic confidence interval construction in the `sparse' case, that also applies to the smoothly clipped absolute deviation estimator, is discussed. The results for the known-variance case are shown to carry over to the unknown-variance case in an appropriate asymptotic sense.

second revision: new title, some comments added, proofs moved to appendix

References in corpus (3)

• Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator
• Confidence Sets Based on Sparse Estimators Are Necessarily Large
• On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding

Cited by in corpus (12)

• Valid post-selection inference
• Exact post-selection inference, with application to the lasso
• On Various Confidence Intervals Post-Model-Selection
• Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
• Uniformly Valid Confidence Sets Based on the Lasso
• Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models
• Confidence Sets Based on Thresholding Estimators in High-Dimensional Gaussian Regression Models
• More Powerful and General Selective Inference for Stepwise Feature Selection using the Homotopy Continuation Approach
• Admissibility of the usual confidence set for the mean of a univariate or bivariate normal population: The unknown-variance case
• Uniformly Valid Inference Based on the Lasso in Linear Mixed Models
• Variable-width confidence intervals in Gaussian regression and penalized maximum likelihood estimators
• Uniform Asymptotics and Confidence Regions Based on the Adaptive Lasso with Partially Consistent Tuning