Optimal learning with -aggregation
arXiv:1301.6080 · doi:10.1214/13-AOS1190
Abstract
We consider a general supervised learning problem with strongly convex and Lipschitz loss and study the problem of model selection aggregation. In particular, given a finite dictionary functions (learners) together with the prior, we generalize the results obtained by Dai, Rigollet and Zhang [Ann. Statist. 40 (2012) 1878-1905] for Gaussian regression with squared loss and fixed design to this learning setup. Specifically, we prove that the -aggregation procedure outputs an estimator that satisfies optimal oracle inequalities both in expectation and with high probability. Our proof techniques somewhat depart from traditional proofs by making most of the standard arguments on the Laplace transform of the empirical process to be controlled.
arXiv commentPublished in at http://dx.doi.org/10.1214/13-AOS1190 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
References in corpus (13)
- Aggregation for Gaussian regression
- Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
- Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
- Fast learning rates in statistical inference through aggregation
- Learning by mirror averaging
- Sparse Estimation by Exponential Weighting
- Deviation optimal learning using greedy Q-aggregation
- Kullback-Leibler aggregation and misspecified generalized linear models
- Sharp Oracle Inequalities for Aggregation of Affine Estimators
- Mirror averaging with sparsity priors
- Pac-bayesian bounds for sparse regression estimation with exponential weights
- Optimal rates of aggregation in classification under low noise assumption
- Sharper lower bounds on the performance of the empirical risk minimization algorithm
Cited by in corpus (6)
- Empirical entropy, minimax regret and minimax risk
- Performance of empirical risk minimization in linear aggregation
- Optimal bounds for aggregation of affine estimators
- Robust angle-based transfer learning in high dimensions
- Optimal exponential bounds for aggregation of density estimators
- An adaptive multiclass nearest neighbor classifier