activity
20232026
most citedSmall-scale proxies for large-scale Transformer training instabilities

4 citations · 5 across the 3 of their papers we have counts for

collaborators
Showing 2026Show all

1 paper · 1 filter

cs.LG2026

Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth

Katie Everett

We investigate how each component of the Transformer feedforward block architecture design determines how much rank survives across depth at initialization. We reinterpret skip con…