XKV: Personalized KV Cache Memory Reduction for Long-Context LLM Inference

Weizhuo Li, Zhigang Wang, Yu Gu +1

Recently the generative Large Language Model (LLM) has achieved remarkable success in numerous applications. Notably its inference generates output tokens one-by-one, leading to ma…