Outlier Robust Online Learning
arXiv:1701.00251
Abstract
We consider the problem of learning from noisy data in practical settings where the size of data is too large to store on a single machine. More challenging, the data coming from the wild may contain malicious outliers. To address the scalability and robustness issues, we present an online robust learning (ORL) approach. ORL is simple to implement and has provable robustness guarantee -- in stark contrast to existing online learning approaches that are generally fragile to outliers. We specialize the ORL approach for two concrete cases: online robust principal component analysis and online linear regression. We demonstrate the efficiency and robustness advantages of ORL through comprehensive simulations and predicting image tags on a large-scale data set. We also discuss extension of the ORL to distributed learning and provide experimental evaluations.
References in corpus (6)
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization
- Geometric median and robust estimation in Banach spaces
- Loss minimization and parameter estimation with heavy tails
- Efficient and fast estimation of the geometric median in Hilbert spaces with an averaged stochastic gradient algorithm
- Robust empirical mean Estimators