1 citations · 1 across the 5 of their papers we have counts for
Showing cs.AIShow all
2 papers · 1 filter
cs.AI2026
Where does Absolute Position come from in decoder-only Transformers?
Valeria Ruscio, Umberto Nanni, Fabrizio Silvestri
RoPE-trained transformers distinguish absolute position in their attention patterns, even though RoPE encodes only relative offsets in the inner product. We trace this leakage to t…
cs.AI2026
On the Tip of the Tongue: Why LLMs Hallucinate Answers They Can Decode
Valeria Ruscio, Keiran Thompson
A language model can give the wrong answer even when the correct answer is decodable from its intermediate states. To study this gap between decodability and selection, we distingu…