68 citations · 160 across the 28 of their papers we have counts for
Showing 2025Show all
2 papers · 1 filter
stat.ML2025
A Proof of Learning Rate Transfer under P
Soufiane Hayou
We provide the first proof of learning rate transfer with width in a linear multi-layer perceptron (MLP) parametrized with P, a neural network parameterization designed to ``max…
cs.LG2025
PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
Soufiane Hayou, Nikhil Ghosh, Bin Yu
Low-Rank Adaptation (LoRA) is a widely used finetuning method for large models. Its small memory footprint allows practitioners to adapt large models to specific tasks at a fractio…