Optimal exponential bounds for aggregation of density estimators
arXiv:1405.3907 · doi:10.3150/15-BEJ742
Abstract
We consider the problem of model selection type aggregation in the context of density estimation. We first show that empirical risk minimization is sub-optimal for this problem and it shares this property with the exponential weights aggregate, empirical risk minimization over the convex hull of the dictionary functions, and all selectors. Using a penalty inspired by recent works on the -aggregation procedure, we derive a sharp oracle inequality in deviation under a simple boundedness assumption and we show that the rate is optimal in a minimax sense. Unlike the procedures based on exponential weights, this estimator is fully adaptive under the uniform prior. In particular, its construction does not rely on the sup-norm of the unknown density. By providing lower bounds with exponential tails, we show that the deviation term appearing in the sharp oracle inequalities cannot be improved.
arXiv commentPublished at http://dx.doi.org/10.3150/15-BEJ742 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)
References in corpus (11)
- Learning by mirror averaging
- Sparse Estimation by Exponential Weighting
- Deviation optimal learning using greedy Q-aggregation
- Kullback-Leibler aggregation and misspecified generalized linear models
- Sharp Oracle Inequalities for Aggregation of Affine Estimators
- Mirror averaging with sparsity priors
- Optimal learning with -aggregation
- Empirical risk minimization is optimal for the convex aggregation problem
- On the optimality of the aggregate with exponential weights for low temperatures
- Lower bounds and aggregation in density estimation
- Sharper lower bounds on the performance of the empirical risk minimization algorithm