Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
A Universal Context-Reuse Layer for Cross-Model KV Sharing
Yi Li, Dongming Jiang, Yi Zhao +1
Modern large language model (LLM) serving systems increasingly operate over repeated or shared context, yet each model typically performs its own prefill computation even when anot…
cs.LG2025
SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
Yi Zhao, Yajuan Peng, Cam-Tu Nguyen +4
KV cache eviction has emerged as an effective solution to alleviate resource constraints faced by LLMs in long-context scenarios. However, existing token-level eviction methods oft…