Disentangling Geometry, Performance, and Training in Language Models

Atharva Kulkarni, Jacob Mitchell Springer, Arjun Subramonian +1

Geometric properties of Transformer weights, particularly the unembedding matrix, have been widely useful in language model interpretability research. Yet, their utility for estima…