collaborators

1 paper

cs.LG2026

Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View

Mingyu Li

Modern Transformer architectures frequently employ normalization mechanisms such as RMSNorm and Query-Key Normalization, making parts of the model approximately scale-invariant wit…