Can one estimate the conditional distribution of post-model-selection estimators?

Abstract

We consider the problem of estimating the conditional distribution of a post-model-selection estimator where the conditioning is on the selected model. The notion of a post-model-selection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model selection criterion such as AIC or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by least-squares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate this distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for this distribution. Similar impossibility results are also obtained for the conditional distribution of linear functions (e.g., predictors) of the post-model-selection estimator.

Published at http://dx.doi.org/10.1214/009053606000000821 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• The distribution of a linear predictor after model selection: Unconditional finite-sample distributions and asymptotic approximations

Cited by in corpus (15)

• Valid post-selection inference
• Exact post-selection inference, with application to the lasso
• Focused information criterion and model averaging for generalized additive partial linear models
• Confidence intervals for high-dimensional inverse covariance estimation
• On adaptive inference and confidence bands
• On the uniform asymptotic validity of subsampling and the bootstrap
• Small area estimation of the homeless in Los Angeles: An application of cost-sensitive stochastic gradient boosting
• On Various Confidence Intervals Post-Model-Selection
• Parametric or nonparametric? A parametricness index for model selection
• Concentration Inequalities and Confidence Bands for Needlet Density Estimators on Compact Homogeneous Manifolds
• Conditional predictive inference post model selection
• Uncertainty Quantification for Matrix Compressed Sensing and Quantum Tomography Problems
• On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding
• A Variational View on Statistical Multiscale Estimation
• On the Distribution of the Adaptive LASSO Estimator