5 papers
SkillMem: Compact External Skill Memory for Latent Multi-Agent Reasoning
Yuxuan Zhu, Mohammad Mohammadi Amiri, Pin-Yu Chen +1
Latent multi-agent systems reduce inference cost by keeping intermediate reasoning and inter-agent communication in continuous representations.\ Reusable skills, however, are commo…
FAD-KV: Adaptive Dictionary KV Caching for Efficient Reasoning Decoding
Yuxuan Zhu, David H. Yang, Parisa Shiravani +5
Key-value (KV) caching is a core mechanism in large language models, where intermediate key and value representations are stored to accelerate autoregressive decoding. As generatio…
ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval
David H. Yang, Yuxuan Zhu, Mohammad Mohammadi Amiri +4
Large language models (LLMs) have shown great performance on complex reasoning tasks but often require generating long intermediate thoughts before reaching a final answer. During…
OjaKV: Context-Aware Online Low-Rank KV Cache Compression
Yuxuan Zhu, David H. Yang, Mohammad Mohammadi Amiri +3
The expanding long-context capabilities of large language models are constrained by a significant memory bottleneck: the key-value (KV) cache required for autoregressive generation…
SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching
Yuxuan Zhu, Ali Falahati, David H. Yang +1
Large language models face significant computational and memory challenges when processing long contexts. During inference, efficient management of the key-value (KV) cache, which…