Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
OjaKV: Context-Aware Online Low-Rank KV Cache Compression
Yuxuan Zhu, David H. Yang, Mohammad Mohammadi Amiri +3
The expanding long-context capabilities of large language models are constrained by a significant memory bottleneck: the key-value (KV) cache required for autoregressive generation…
cs.CL2025
SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching
Yuxuan Zhu, Ali Falahati, David H. Yang +1
Large language models face significant computational and memory challenges when processing long contexts. During inference, efficient management of the key-value (KV) cache, which…