Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
Yuhang Zhan, Lisi Chen, Shuo Shang
KV cache compression is essential for efficient long-context inference. Existing eviction methods permanently discard unselected tokens and consequently remove their aggregate cont…
cs.CL2024
Not All Layers of LLMs Are Necessary During Inference
Siqi Fan, Xin Jiang, Xiang Li +6
Due to the large number of parameters, the inference phase of Large Language Models (LLMs) is resource-intensive. However, not all requests posed to LLMs are equally difficult to h…