On Hodges' Superefficiency and Merits of Oracle Property in Model Selection

Abstract

The oracle property of model selection procedures has attracted a large volume of favorable publications in the literature, but also faced criticisms of being ineffective and misleading in applications. In this paper, we introduce a class of estimators that can easily produce model selection procedures possessing the oracle property and discuss the merits of the oracle property by analyzing the performance of such estimators in finite sample size theoretically. Specifically, we propose a new type of Hodges' estimators capable of reducing the asymptotic variance of any given estimator over a multi-dimensional subspace of the parameter space, which can easily produce model selection procedures with the oracle and some other desired properties. This new type of oracle estimators, however, perform poorly at some values of the parameters for estimation, and there is no convincing reason to declare that oracle estimators are better than the traditional estimators such as the MLE and LSE. Consequently, the merits of the oracle property for model selection as claimed in the literature are probably grossly overstated, and the criticisms of the oracle property are justifiable.

arXiv comment29 pages, 1 fiture

References in corpus (10)

• Nearly unbiased variable selection under minimax concave penalty
• On the adaptive elastic-net with a diverging number of parameters
• Valid post-selection inference
• Variable selection in nonparametric additive models
• High-dimensional generalized linear models and the lasso
• Network exploration via the adaptive LASSO and SCAD penalties
• Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator
• Random lasso
• Confidence Sets Based on Sparse Estimators Are Necessarily Large
• On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding