12 citations · 22 across the 9 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023
Saddle-to-Saddle Dynamics in Diagonal Linear Networks
Scott Pesme, Nicolas Flammarion
In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively…
cs.LG2023★ 2 cited
(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability
Mathieu Even, Scott Pesme, Suriya Gunasekar +1
In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over dia…