35 citations · 81 across the 22 of their papers we have counts for
22 papers
Minimax-Optimal Early Stopping for SGD via the Discrepancy Principle
Tim Jahn, Loucas Pillaud-Vivien, Adrien Schertzer
We study early stopping for stochastic gradient descent with a constant step size applied to noisy linear inverse problems with polynomially decaying spectrum. The iteration is sto…
Incremental Learning in Mirror Flows
Raphaël Berthier, Loucas Pillaud-Vivien
We study mirror flows generated by a convex quadratic loss and a general convex lower semicontinuous mirror potential. We show that, when initialized near the boundary of the domai…
Oscillating solutions to the mean-field Langevin descent-ascent flow
Jean-Christophe Mourrat, Loucas Pillaud-Vivien
We present a counterexample to the statement of convergence of the mean-field Langevin descent-ascent flow on . We consider payoff functions that are shaped as a doub…
Joint Learning in the Gaussian Single Index Model
Loucas Pillaud-Vivien, Adrien Schertzer
We consider the problem of jointly learning a one-dimensional projection and a univariate function in high-dimensional Gaussian models. Specifically, we study predictors of the for…
Convergence of Shallow ReLU Networks on Weakly Interacting Data
Léo Dana, Francis Bach, Loucas Pillaud-Vivien
We analyse the convergence of one-hidden-layer ReLU networks trained by gradient flow on data points. Our main contribution leverages the high dimensionality of the ambient spa…
Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descent
Adrien Schertzer, Loucas Pillaud-Vivien
We study the dynamics of a continuous-time model of the Stochastic Gradient Descent (SGD) for the least-square problem. Indeed, pursuing the work of Li et al. (2019), we analyze St…
Computational-Statistical Gaps in Gaussian Single-Index Models
Alex Damian, Loucas Pillaud-Vivien, Jason D. Lee +1
Single-Index Models are high-dimensional regression problems with planted structure, whereby labels depend on an unknown one-dimensional projection of the input via a generic, non-…
Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs
Charles C. Margossian, Loucas Pillaud-Vivien, Lawrence K. Saul
Given an intractable distribution , the problem of variational inference (VI) is to find the best approximation from some more tractable family . Commonly, one chooses to…
Batch and match: black-box variational inference with a score-based divergence
Diana Cai, Chirag Modi, Loucas Pillaud-Vivien +4
Most leading implementations of black-box variational inference (BBVI) are based on optimizing a stochastic evidence lower bound (ELBO). But such approaches to BBVI often converge…
On Learning Gaussian Multi-index Models with Gradient Flow
Alberto Bietti, Joan Bruna, Loucas Pillaud-Vivien
We study gradient flow on the multi-index regression problem for high-dimensional Gaussian data. Multi-index functions consist of a composition of an unknown low-rank linear projec…
On Single Index Models beyond Gaussian Data
Joan Bruna, Loucas Pillaud-Vivien, Aaron Zweig
Sparse high-dimensional functions have arisen as a rich framework to study the behavior of gradient-descent methods using shallow neural networks, showcasing their ability to perfo…
Kernelized Diffusion maps
Loucas Pillaud-Vivien, Francis Bach
Spectral clustering and diffusion maps are celebrated dimensionality reduction algorithms built on eigen-elements related to the diffusive structure of the data. The core of these…
SGD with Large Step Sizes Learns Sparse Features
Maksym Andriushchenko, Aditya Varre, Loucas Pillaud-Vivien +1
We showcase important features of the dynamics of the Stochastic Gradient Descent (SGD) in the training of neural networks. We present empirical observations that commonly used lar…
Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation
Loucas Pillaud-Vivien, Julien Reygner, Nicolas Flammarion
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the rol…
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs
Etienne Boursier, Loucas Pillaud-Vivien, Nicolas Flammarion
The training of neural networks by gradient descent methods is a cornerstone of the deep learning revolution. Yet, despite some recent progress, a complete theory explaining its su…
Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Scott Pesme, Loucas Pillaud-Vivien, Nicolas Flammarion
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dyn…
Last iterate convergence of SGD for Least-Squares in the Interpolation regime
Aditya Varre, Loucas Pillaud-Vivien, Nicolas Flammarion
Motivated by the recent successes of neural networks that have the ability to fit the data perfectly and generalize well, we study the noiseless model in the fundamental least-squa…
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning
Vivien Cabannes, Loucas Pillaud-Vivien, Francis Bach +1
As annotations of data can be scarce in large-scale practical problems, leveraging unlabelled examples is one of the most important aspects of machine learning. This is the aim of…
Statistical Estimation of the Poincar{é} constant and Application to Sampling Multimodal Distributions
Loucas Pillaud-Vivien, Francis Bach, Tony Lelièvre +2
Poincar{é} inequalities are ubiquitous in probability and analysis and have various applications in statistics (concentration of measure, rate of convergence of Markov chains). The…
Central Limit Theorem for stationary Fleming--Viot particle systems in finite spaces
Tony Lelievre, Loucas Pillaud-Vivien, Julien Reygner
We consider the Fleming--Viot particle system associated with a continuous-time Markov chain in a finite space. Assuming irreducibility, it is known that the particle system posses…
Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes
Loucas Pillaud-Vivien, Alessandro Rudi, Francis Bach
We consider stochastic gradient descent (SGD) for least-squares regression with potentially several passes over the data. While several passes have been widely reported to perform…
Exponential convergence of testing error for stochastic gradient methods
Loucas Pillaud-Vivien, Alessandro Rudi, Francis Bach
We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the ex…