Learning with risks based on M-location
arXiv:2012.02424 · doi:10.1007/s10994-022-06217-5
Abstract
In this work, we study a new class of risks defined in terms of the location and deviation of the loss distribution, generalizing far beyond classical mean-variance risk functions. The class is easily implemented as a wrapper around any smooth loss, it admits finite-sample stationarity guarantees for stochastic gradient methods, it is straightforward to interpret and adjust, with close links to M-estimators of the loss location, and has a salient effect on the test loss distribution.
Substantial update to initial version; refined theory, improved exposition, added experimental analysis
References in corpus (4)
- A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets
- Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes
- Empirical Bernstein Bounds and Sample Variance Penalization
- DORO: Distributional and Outlier Robust Optimization