374 citations · 941 across the 27 of their papers we have counts for
Showing 2025Show all
2 papers · 1 filter
cs.LG2025
On the Importance of Gaussianizing Representations
Daniel Eftekhari, Vardan Papyan
The normal distribution plays a central role in information theory - it is at the same time the best-case signal and worst-case noise distribution, has the greatest representationa…
cs.CL2025
Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
Stephen Zhang, Mustafa Khan, Vardan Papyan
Large language models (LLMs) often concentrate their attention on a few specific tokens referred to as attention sinks. Common examples include the first token, a prompt-independen…