CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing

Yixuan Wang, Haoyu Qiao, Lujun Li +2

Large Language Models (LLMs) confront significant memory challenges due to the escalating KV cache with increasing sequence length. As a crucial technique, existing cross-layer KV…