2 papers
cs.LG2025
LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation
Yiqun Shen, Song Yuan, Zhengze Zhang +3
KV Cache is commonly used to accelerate LLM inference with long contexts, yet its high memory demand drives the need for cache compression. Existing compression methods, however, a…
cs.LG2025
daDPO: Distribution-Aware DPO for Distilling Conversational Abilities
Zhengze Zhang, Shiqi Wang, Yiqun Shen +5
Large language models (LLMs) have demonstrated exceptional performance across various applications, but their conversational abilities decline sharply as model size decreases, pres…