3 citations · 6 across the 9 of their papers we have counts for
3 papers · 1 filter
The Feature Speed Formula: a flexible approach to scale hyper-parameters of deep neural networks
Lénaïc Chizat, Praneeth Netrapalli
Deep learning succeeds by doing hierarchical feature learning, yet tuning hyper-parameters (HP) such as initialization scales, learning rates etc., only give indirect control over…
On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks
Sebastian Neumayer, Lénaïc Chizat, Michael Unser
In supervised learning, the regularization path is sometimes used as a convenient theoretical proxy for the optimization path of gradient descent initialized from zero. In this pap…
Training Integrable Parameterizations of Deep Neural Networks in the Infinite-Width Limit
Karl Hajjar, Lénaïc Chizat, Christophe Giraud
To theoretically understand the behavior of trained deep neural networks, it is necessary to study the dynamics induced by gradient methods from a random initialization. However, t…