2 papers
cs.LG2026
Key-Value Means: Transformers with Expandable Block-Recurrent Compressed Memory
Daniel Goldstein, Eugene Cheah, Navneel Singhal
Recall presents a difficult choice: transformers have a linearly growing memory that slows each successive token, while linear RNNs typically have fixed costs but limited recall. W…
cs.CL2026
RADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale
Daniel Goldstein, Eric Alcaide, Janna Lu +1
We present Rapid Attention Distillation to Linear Attention Decoders at Scale (RADLADS), a protocol for rapidly converting softmax attention transformers into linear attention deco…