Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Augmenting Attention with Exponentially Decaying Memory Improves Query-Aware KV Sparsity
Xiuying Wei, Caglar Gulcehre
Efficient inference is critical for long-context language models, where attention computation and KV-cache access dominate the cost. Recent work RAT+, introduces a recurrence-augme…
cs.LG2026
RAT+: Train Dense, Infer Sparse -- Recurrence Augmented Attention for Dilated Inference
Xiuying Wei, Caglar Gulcehre
Structured dilated attention has an appealing inference-time efficiency knob: it reduces the FLOPs of attention and the KV cache size by a factor of the dilation size D, while pres…
cs.LG2025
From Markov to Laplace: How Mamba In-Context Learns Markov Chains
Marco Bondaschi, Nived Rajaraman, Xiuying Wei +5
While transformer-based language models have driven the AI revolution thus far, their computational complexity has spurred growing interest in viable alternatives, such as structur…