2 papers
cs.LG2026
An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference
Feiyu Yao, Zhixiong Niu, Xiaqing Li +3
Long-context inference increasingly operates over CPU-resident KV caches, either because decoding-time KV states exceed GPU memory capacity or because disaggregated prefill-decode…
cs.LG2025
Learn from the Past: Fast Sparse Indexing for Large Language Model Decoding
Feiyu Yao, Qian Wang
As large language models (LLMs) continue to support increasingly longer contexts, the memory demand for key-value (KV) caches during decoding grows rapidly, becoming a critical bot…