40 citations · 106 across the 28 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023★ 2 cited
Length Generalization in Arithmetic Transformers
Samy Jelassi, Stéphane d'Ascoli, Carles Domingo-Enrich +3
We examine how transformers cope with two challenges: learning basic integer arithmetic, and generalizing to longer sequences than seen during training. We find that relative posit…
cs.LG2023
Depth Dependence of P Learning Rates in ReLU MLPs
Samy Jelassi, Boris Hanin, Ziwei Ji +3
In this short note we consider random fully connected ReLU networks of width and depth equipped with a mean-field weight initialization. Our purpose is to study the depende…