Parametric or nonparametric? A parametricness index for model selection

Abstract

In model selection literature, two classes of criteria perform well asymptotically in different situations: Bayesian information criterion (BIC) (as a representative) is consistent in selection when the true model is finite dimensional (parametric scenario); Akaike's information criterion (AIC) performs well in an asymptotic efficiency when the true model is infinite dimensional (nonparametric scenario). But there is little work that addresses if it is possible and how to detect the situation that a specific model selection problem is in. In this work, we differentiate the two scenarios theoretically under some conditions. We develop a measure, parametricness index (PI), to assess whether a model selected by a potentially consistent procedure can be practically treated as the true model, which also hints on AIC or BIC is better suited for the data for the goal of estimating the regression function. A consequence is that by switching between AIC and BIC based on the PI, the resulting regression estimator is simultaneously asymptotically efficient for both parametric and nonparametric scenarios. In addition, we systematically investigate the behaviors of PI in simulation and real data and show its usefulness.

Published in at http://dx.doi.org/10.1214/11-AOS899 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (10)

• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Fisher Lecture: Dimension Reduction in Regression
• A unified approach to model selection and sparse recovery using regularized least squares
• Consistency of cross validation for comparing regression procedures
• Can one estimate the conditional distribution of post-model-selection estimators?
• Order selection for same-realization predictions in autoregressive processes
• Rejoinder: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Accumulated prediction errors, information criteria and optimal forecasting for autoregressive time series
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• Catching Up Faster by Switching Sooner: A Prequential Solution to the AIC-BIC Dilemma