13 citations · 16 across the 10 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.DC2024★ 13 cited
Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
Ruoyu Qin, Zheming Li, Weiran He +4
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. It features a KVCache-centric disaggregated architecture that separates the prefill and de…
cs.LG2024★ 1 cited
Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
Shaoyuan Chen, Wencong Xiao, Yutong Lin +5
Transformer-based large language models (LLMs) exhibit impressive performance in generative tasks but also introduce significant challenges in real-world serving due to inefficient…