8 citations · 10 across the 15 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
Geometry and Dynamics of LayerNorm
Paul M. Riechers
A technical note aiming to offer deeper intuition for the LayerNorm function common in deep neural networks. LayerNorm is defined relative to a distinguished 'neural' basis, but it…
cs.LG2024
Transformers represent belief state geometry in their residual stream
Adam S. Shai, Sarah E. Marzen, Lucas Teixeira +2
What computational structure are we building into large language models when we train them on next-token prediction? Here, we present evidence that this structure is given by the m…