Statistical learning with Lipschitz and convex loss functions

Abstract

We obtain risk bounds for Empirical Risk Minimizers (ERM) and minmax Median-Of-Means (MOM) estimators based on loss functions that are both Lipschitz and convex. Results for the ERM are derived without assumptions on the outputs and under subgaussian assumptions on the design and a new "local Bernstein assumption" on the class of predictors. Similar results are shown for minmax MOM estimators in a close setting where the design is only supposed to satisfy moment assumptions, relaxing the Subgaussian hypothesis necessary for ERM. The analysis of minmax MOM estimators is not based on the small ball assumption (SBA) as it was the case in the first analysis of minmax MOM estimators. In particular, the basic example of non parametric statistics where the learning class is the linear span of localized bases, that does not satisfy SBA can now be handled. Finally, minmax MOM estimators are analysed in a setting where the local Bernstein condition is also dropped out. It is shown to achieve an oracle inequality with exponentially large probability under minimal assumptions insuring the existence of all objects.

References in corpus (2)

• Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions
• Convergence rates of least squares regression estimators with heavy-tailed errors

Cited by in corpus (5)

• Excess risk bounds in robust empirical risk minimization
• A spectral algorithm for robust regression with subgaussian rates
• Algorithms of Robust Stochastic Optimization Based on Mirror Descent Method
• Learning with Semi-Definite Programming: new statistical bounds based on fixed point analysis and excess risk curvature
• Benign overfitting without concentration