Honest confidence sets for high-dimensional regression by projection and shrinkage

Abstract

The issue of honesty in constructing confidence sets arises in nonparametric regression. While optimal rate in nonparametric estimation can be achieved and utilized to construct sharp confidence sets, severe degradation of confidence level often happens after estimating the degree of smoothness. Similarly, for high-dimensional regression, oracle inequalities for sparse estimators could be utilized to construct sharp confidence sets. Yet the degree of sparsity itself is unknown and needs to be estimated, causing the honesty problem. To resolve this issue, we develop a novel method to construct honest confidence sets for sparse high-dimensional linear regression. The key idea in our construction is to separate signals into a strong and a weak group, and then construct confidence sets for each group separately. This is achieved by a projection and shrinkage approach, the latter implemented via Stein estimation and the associated Stein unbiased risk estimate. Our confidence set is honest over the full parameter space without any sparsity constraints, while its diameter adapts to the optimal rate of $n^{-1/4}$ when the true parameter is indeed sparse. Through extensive numerical comparisons, we demonstrate that our method outperforms other competitors with big margins for finite samples, including oracle methods built upon the true sparsity of the underlying model.

arXiv comment36 pages, 7 figures

References in corpus (17)

• Nearly unbiased variable selection under minimax concave penalty
• Simultaneous analysis of Lasso and Dantzig selector
• On asymptotically optimal confidence regions and tests for high-dimensional models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism
• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• Confidence sets in sparse regression
• Adaptive nonparametric confidence sets
• Adaptive confidence balls
• Confidence sets for nonparametric wavelet regression
• Confidence balls in Gaussian regression
• Testability of high-dimensional linear models with non-sparse structures
• Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
• Uniformly Valid Confidence Sets Based on the Lasso
• Confidence Sets Based on Thresholding Estimators in High-Dimensional Gaussian Regression Models
• Constructing confidence sets after lasso selection by randomized estimator augmentation
• Estimator Augmentation with Applications in High-Dimensional Group Inference