CaliDrop: KV Cache Compression with Calibration

Yi Su, Quantong Qiu, Yuechi Zhou +6

Large Language Models (LLMs) require substantial computational resources during generation. While the Key-Value (KV) cache significantly accelerates this process by storing attenti…