3 citations · 6 across the 9 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023
The Feature Speed Formula: a flexible approach to scale hyper-parameters of deep neural networks
Lénaïc Chizat, Praneeth Netrapalli
Deep learning succeeds by doing hierarchical feature learning, yet tuning hyper-parameters (HP) such as initialization scales, learning rates etc., only give indirect control over…
cs.LG2023
On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks
Sebastian Neumayer, Lénaïc Chizat, Michael Unser
In supervised learning, the regularization path is sometimes used as a convenient theoretical proxy for the optimization path of gradient descent initialized from zero. In this pap…