Performance of empirical risk minimization in linear aggregation
arXiv:1402.5763 · doi:10.3150/15-BEJ701
Abstract
We study conditions under which, given a dictionary and an i.i.d. sample , the empirical minimizer in relative to the squared loss, satisfies that with high probability \[R\bigl(\tilde{f}^{\mathrm{ERM}}\bigr)\leq\inf_{f\in\operatorname {span}(F)}R(f)+r_N(M),\] where is the squared risk and is of the order of . Among other results, we prove that a uniform small-ball estimate for functions in is enough to achieve that goal when the noise is independent of the design.
arXiv commentPublished at http://dx.doi.org/10.3150/15-BEJ701 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)
References in corpus (9)
- Aggregation for Gaussian regression
- Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
- Robust linear least squares regression
- Learning by mirror averaging
- Learning subgaussian classes : Upper and minimax bounds
- Optimal learning with -aggregation
- Empirical risk minimization is optimal for the convex aggregation problem
- On the optimality of the aggregate with exponential weights for low temperatures
- Sharper lower bounds on the performance of the empirical risk minimization algorithm