46 citations · 137 across the 23 of their papers we have counts for
15 papers · 1 filter
Robust Stochastic Optimization via Gradient Quantile Clipping
Ibrahim Merad, Stéphane Gaïffas
We introduce a clipping strategy for Stochastic Gradient Descent (SGD) which uses quantiles of the gradient norm as clipping thresholds. We prove that this new strategy provides a…
Convergence and concentration properties of constant step-size SGD through Markov chains
Ibrahim Merad, Stéphane Gaïffas
We consider the optimization of a smooth and strongly convex objective using constant step-size stochastic gradient descent (SGD) and study its properties through the prism of Mark…
Robust Methods for High-Dimensional Linear Learning
Ibrahim Merad, Stéphane Gaïffas
We propose statistically robust and computationally efficient linear learning methods in the high-dimensional batch setting, where the number of features may exceed the sample…
Robust supervised learning with coordinate gradient descent
Stéphane Gaïffas, Ibrahim Merad
This paper considers the problem of supervised learning with linear methods when both features and labels can be corrupted, either in the form of heavy tailed data and/or corrupted…
AMF: Aggregated Mondrian Forests for Online Learning
Jaouad Mourtada, Stéphane Gaïffas, Erwan Scornet
Random Forests (RF) is one of the algorithms of choice in many supervised learning applications, be it classification or regression. The appeal of such tree-ensemble methods comes…
On the optimality of the Hedge algorithm in the stochastic regime
Jaouad Mourtada, Stéphane Gaïffas
In this paper, we study the behavior of the Hedge algorithm in the online stochastic setting. We prove that anytime Hedge with decreasing learning rate, which is one of the simples…