2 citations · 2 across the 1 of their papers we have counts for
6 papers
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs
Etienne Boursier, Loucas Pillaud-Vivien, Nicolas Flammarion
The training of neural networks by gradient descent methods is a cornerstone of the deep learning revolution. Yet, despite some recent progress, a complete theory explaining its su…
Early alignment in two-layer networks training is a two-edged sword
Etienne Boursier, Nicolas Flammarion
Training neural networks with first order optimisation methods is at the core of the empirical success of deep learning. The scale of initialisation is a crucial factor, as small i…
Long-Context Linear System Identification
OÄuz Kaan Yüksel, Mathieu Even, Nicolas Flammarion
This paper addresses the problem of long-context linear system identification, where the state of a dynamical system at time depends linearly on previous states ove…
Simplicity bias and optimization threshold in two-layer ReLU networks
Etienne Boursier, Nicolas Flammarion
Understanding generalization of overparametrized neural networks remains a fundamental challenge in machine learning. Most of the literature mostly studies generalization from an i…
Penalising the biases in norm regularisation enforces sparsity
Etienne Boursier, Nicolas Flammarion
Controlling the parameters' norm often yields good generalisation when training neural networks. Beyond simple intuitions, the relation between regularising parameters' norm and ob…
Implicit Bias of Mirror Flow on Separable Data
Scott Pesme, Radu-Alexandru Dragomir, Nicolas Flammarion
We examine the continuous-time counterpart of mirror descent, namely mirror flow, on classification problems which are linearly separable. Such problems are minimised `at infinity'…