HGCA: Hybrid GPU-CPU Attention for Long Context LLM Inference

Weishu Deng, Yujie Yang, Peiran Du +6

Scaling inference for large language models (LLMs) is increasingly constrained by limited GPU memory, especially due to growing key-value (KV) caches required for long-context gene…