35 citations · 81 across the 22 of their papers we have counts for
9 papers · 1 filter
Joint Learning in the Gaussian Single Index Model
Loucas Pillaud-Vivien, Adrien Schertzer
We consider the problem of jointly learning a one-dimensional projection and a univariate function in high-dimensional Gaussian models. Specifically, we study predictors of the for…
Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descent
Adrien Schertzer, Loucas Pillaud-Vivien
We study the dynamics of a continuous-time model of the Stochastic Gradient Descent (SGD) for the least-square problem. Indeed, pursuing the work of Li et al. (2019), we analyze St…
Computational-Statistical Gaps in Gaussian Single-Index Models
Alex Damian, Loucas Pillaud-Vivien, Jason D. Lee +1
Single-Index Models are high-dimensional regression problems with planted structure, whereby labels depend on an unknown one-dimensional projection of the input via a generic, non-…
On Single Index Models beyond Gaussian Data
Joan Bruna, Loucas Pillaud-Vivien, Aaron Zweig
Sparse high-dimensional functions have arisen as a rich framework to study the behavior of gradient-descent methods using shallow neural networks, showcasing their ability to perfo…
SGD with Large Step Sizes Learns Sparse Features
Maksym Andriushchenko, Aditya Varre, Loucas Pillaud-Vivien +1
We showcase important features of the dynamics of the Stochastic Gradient Descent (SGD) in the training of neural networks. We present empirical observations that commonly used lar…
Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Scott Pesme, Loucas Pillaud-Vivien, Nicolas Flammarion
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dyn…
Last iterate convergence of SGD for Least-Squares in the Interpolation regime
Aditya Varre, Loucas Pillaud-Vivien, Nicolas Flammarion
Motivated by the recent successes of neural networks that have the ability to fit the data perfectly and generalize well, we study the noiseless model in the fundamental least-squa…
Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes
Loucas Pillaud-Vivien, Alessandro Rudi, Francis Bach
We consider stochastic gradient descent (SGD) for least-squares regression with potentially several passes over the data. While several passes have been widely reported to perform…
Exponential convergence of testing error for stochastic gradient methods
Loucas Pillaud-Vivien, Alessandro Rudi, Francis Bach
We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the ex…