The Geometry of Language Model Logits: Generic Structure and Learned Departures

Swathi Shree Narashiman, Aravind Ramana Venkatramanan, Nambi Seshadri

Autoregressive language models produce, at every prediction step, a vocabulary-sized logit vector whose structure determines the conditional next-token distribution. We empirically…