1 citations · 2 across the 8 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
Yuhang Zhan, Lisi Chen, Shuo Shang
KV cache compression is essential for efficient long-context inference. Existing eviction methods permanently discard unselected tokens and consequently remove their aggregate cont…
cs.CL2024★ 1 cited
Not All Layers of LLMs Are Necessary During Inference
Siqi Fan, Xin Jiang, Xiang Li +6
Due to the large number of parameters, the inference phase of Large Language Models (LLMs) is resource-intensive. However, not all requests posed to LLMs are equally difficult to h…