2 papers
cs.AI2026
Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating
Maruthi Vemula, Neeraj Praneeth Gajula
A language model with a bounded working memory must repeatedly decide which stored items to keep. Every deployed method decides the moment an item arrives, from the past (Streaming…
cs.LG2026
Massive Activations Are Architecturally Robust: A Controlled Scratch/Commitment Residual Stream Test
Maruthi Vemula
Trained transformers reliably develop massive activations, a small number of hidden dimensions whose magnitude is far above the median and which concentrate on the sequence-start t…