Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation
Yiqun Shen, Song Yuan, Zhengze Zhang +3
KV Cache is commonly used to accelerate LLM inference with long contexts, yet its high memory demand drives the need for cache compression. Existing compression methods, however, a…
cs.LG2025
SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
Yi Zhao, Yajuan Peng, Cam-Tu Nguyen +4
KV cache eviction has emerged as an effective solution to alleviate resource constraints faced by LLMs in long-context scenarios. However, existing token-level eviction methods oft…