13 citations · 16 across the 10 of their papers we have counts for
7 papers · 1 filter
AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation
Chengyu Qiu, Xiao Fu, Fengcun Li +6
Attention--Feed-Forward Network (FFN) Disaggregation (AFD) is emerging as a promising architecture for serving Mixture-of-Experts (MoE) language models. While existing AFD systems…
Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
Xun Sun, Shaoyuan Chen, Pingchuan Ma +18
Mixture-of-Experts (MoE) serving relies on wide expert parallelism (EP) to aggregate the memory capacity and bandwidth of many GPUs within one inference instance. This efficiency c…
TENT: A Declarative Slice Spraying Engine for Performant and Resilient Data Movement in Disaggregated LLM Serving
Feng Ren, Ruoyu Qin, Teng Ma +16
Modern GPU clusters rely on complex, heterogeneous interconnects. As large language model (LLM) serving shifts toward agentic reasoning, KVCache becomes a first-class mobile asset,…
Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning
Ruoyu Qin, Weiran He, Weixiao Huang +7
Reinforcement Learning (RL) has emerged as a critical technique for advancing modern Large Language Models (LLMs), yet existing synchronous RL systems face severe performance bottl…
TrEnv-X: Transparently Share Serverless Execution Environments Across Different Functions and Nodes
Jialiang Huang, Teng Ma, Zheng Liu +11
Serverless computing is renowned for its computation elasticity, yet its full potential is often constrained by the requirement for functions to operate within local and dedicated…
Efficient Graph-Based Approximate Nearest Neighbor Search Achieving: Low Latency Without Throughput Loss
Jingjia Luo, Mingxing Zhang, Kang Chen +4
The increase in the dimensionality of neural embedding models has enhanced the accuracy of semantic search capabilities but also amplified the computational demands for Approximate…
Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
Ruoyu Qin, Zheming Li, Weiran He +4
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. It features a KVCache-centric disaggregated architecture that separates the prefill and de…