Robust empirical mean Estimators

Abstract

We study robust estimators of the mean of a probability measure $P$, called robust empirical mean estimators. This elementary construction is then used to revisit a problem of aggregation and a problem of estimator selection, extending these methods to not necessarily bounded collections of previous estimators. We consider then the problem of robust $M$-estimation. We propose a slightly more complicated construction to handle this problem and, as examples of applications, we apply our general approach to least-squares density estimation, to density estimation with Küllback loss and to a non-Gaussian, unbounded, random design and heteroscedastic regression problem. Finally, we show that our strategy can be used when the data are only assumed to be mixing.

47 pages

References in corpus (4)

• Simultaneous analysis of Lasso and Dantzig selector
• Risk bounds for statistical learning
• PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers
• Risk bounds in linear regression through PAC-Bayesian truncation

Cited by in corpus (20)

• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Geometric median and robust estimation in Banach spaces
• Loss minimization and parameter estimation with heavy tails
• Fast Mean Estimation with Sub-Gaussian Rates
• Propose, Test, Release: Differentially private estimation with high probability
• Robust Modifications of U-statistics and Applications to Covariance Estimation Problems
• Sub-Gaussian estimators of the mean of a random vector
• TAdam: A Robust Stochastic Gradient Optimizer
• Excess risk bounds in robust empirical risk minimization
• Outlier Robust Online Learning
• Robust estimation of U-statistics
• Distribution-robust mean estimation via smoothed random perturbations
• A Survey of Learning Criteria Going Beyond the Usual Risk
• Efficient learning with robust gradient descent
• Learning with CVaR-based feedback under potentially heavy tails
• On deviation probabilities in non-parametric regression with heavy-tailed noise
• On the estimation of the mean of a random vector
• A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
• Non-asymptotic analysis and inference for an outlyingness induced winsorized mean
• Robust Covariance Estimation for High-dimensional Compositional Data with Application to Microbial Communities Analysis