Breaking the Boundaries of Long-Context LLM Inference: Adaptive KV Management on a Single Commodity GPU

He Sun, Li Li, Mingjun Xiao +1

Advanced Large Language Models (LLMs) have achieved impressive performance across a wide range of complex and long-context natural language tasks. However, performing long-context…