CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration

Haoyun Jiang, Haolin Li, Jianwei Zhang +7

Large language models (LLMs) have demonstrated strong capabilities in handling long-context tasks, but processing such long contexts remains challenging due to the substantial memo…