PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression

Zizhong Wang, Jieying Wang, Zhao Zhang +1

Long-context inference in large language models (LLMs) is increasingly limited by the memory required for the key-value (KV) cache. KV cache compression addresses this problem by r…