Symmetry in language statistics shapes the geometry of model representations

Dhruva Karkada, Daniel J. Korchinski, Andres Nava +2

The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth on…