A universal procedure for aggregating estimators
arXiv:0704.2500 · doi:10.1214/00-AOS576
Abstract
In this paper we study the aggregation problem that can be formulated as follows. Assume that we have a family of estimators built on the basis of available observations. The goal is to construct a new estimator whose risk is as close as possible to that of the best estimator in the family. We propose a general aggregation scheme that is universal in the following sense: it applies for families of arbitrary estimators and a wide variety of models and global risk measures. The procedure is based on comparison of empirical estimates of certain linear functionals with estimates induced by the family . We derive oracle inequalities and show that they are unimprovable in some sense. Numerical results demonstrate good practical behavior of the procedure.
Published in at http://dx.doi.org/10.1214/00-AOS576 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)