On Various Confidence Intervals Post-Model-Selection

Abstract

We compare several confidence intervals after model selection in the setting recently studied by Berk et al. [Ann. Statist. 41 (2013) 802-837], where the goal is to cover not the true parameter but a certain nonstandard quantity of interest that depends on the selected model. In particular, we compare the PoSI-intervals that are proposed in that reference with the "naive" confidence interval, which is constructed as if the selected model were correct and fixed a priori (thus ignoring the presence of model selection). Overall, we find that the actual coverage probabilities of all these intervals deviate only moderately from the desired nominal coverage probability. This finding is in stark contrast to several papers in the existing literature, where the goal is to cover the true parameter.

arXiv commentPublished at http://dx.doi.org/10.1214/14-STS507 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (19)

• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Valid post-selection inference
• Can one estimate the conditional distribution of post-model-selection estimators?
• Confidence Sets Based on Penalized Maximum Likelihood Estimators in Gaussian Regression
• The distribution of model averaging estimators and an impossibility result regarding its estimation
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• Confidence Sets Based on Sparse Estimators Are Necessarily Large
• The distribution of a linear predictor after model selection: Unconditional finite-sample distributions and asymptotic approximations
• On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding
• Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models

Cited by in corpus (4)

• Simultaneous confidence bands for contrasts between several nonlinear regression curves
• Mixture of Linear Models Co-supervised by Deep Neural Networks
• Confidence intervals centered on bootstrap smoothed estimators
• Admissibility of the usual confidence set for the mean of a univariate or bivariate normal population: The unknown-variance case