Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Retrieval Capacity of Self-Attention Under Competition
Timur Mudarisov, Mikhail Burtsev, Radu State
How many tokens from its context does a language model actually use, and what determines that number? We study this question through self-attention. Without retraining, we retain o…
cs.CL2026
The Geometry of Inference in Transformer Residual Streams
Timur Mudarisov, Mikhail Burtsev, Radu State
Transformer language models build predictions through successive residual updates, but how their representations become specific to an eventual outcome remains unclear. We study th…