2 papers
cs.AI2026
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
Arya Tschand, Yaosheng Fu, Vikram Sharma Mailthody +6
Frontier language models are more aggressively using subquadratic attention to reduce the memory footprint and compute requirements during inference while still delivering frontier…
cs.CL2025
RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
Payman Behnam, Yaosheng Fu, Ritchie Zhao +3
Transformer-based Large Language Models rely critically on the KV cache to efficiently handle extended contexts during the decode phase. Yet, the size of the KV cache grows proport…