13 citations · 17 across the 11 of their papers we have counts for
11 papers
TIDE-MC: Two-Sided Interpolative Decomposition for Billion-Scale GPU Matrix Completion
Chengying Huan, Yubo Wang, Pinhuan Wang +11
Matrix completion supports large-scale recommendation and scientific computing, yet existing GPU solvers commonly assume that the observed matrix or its dense factors fit in device…
Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
Xun Sun, Shaoyuan Chen, Pingchuan Ma +18
Mixture-of-Experts (MoE) serving relies on wide expert parallelism (EP) to aggregate the memory capacity and bandwidth of many GPUs within one inference instance. This efficiency c…
Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter
Ruoyu Qin, Weiran He, Yaoyu Wang +5
Prefill-decode (PD) disaggregation has become the standard architecture for large-scale LLM serving, but in practice its deployment boundary is still determined by KVCache transfer…
TENT: A Declarative Slice Spraying Engine for Performant and Resilient Data Movement in Disaggregated LLM Serving
Feng Ren, Ruoyu Qin, Teng Ma +16
Modern GPU clusters rely on complex, heterogeneous interconnects. As large language model (LLM) serving shifts toward agentic reasoning, KVCache becomes a first-class mobile asset,…
DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
Yongtong Wu, Shaoyuan Chen, Yinmin Zhong +10
The performance of multi-turn, agentic LLM inference is increasingly dominated by KV-Cache storage I/O rather than computation. In prevalent disaggregated architectures, loading th…
OrchANN: Hierarchical Orchestration for Skewed Out-of-Core Vector Search
Lizheng Chen, Pinhuan Wang, Shaonan Ma +9
At billion scale, approximate nearest neighbor search (ANNS) often becomes an out-of-core problem: the full vector collection and index structures exceed memory capacity, making qu…
Efficient Graph-Based Approximate Nearest Neighbor Search Achieving: Low Latency Without Throughput Loss
Jingjia Luo, Mingxing Zhang, Kang Chen +4
The increase in the dimensionality of neural embedding models has enhanced the accuracy of semantic search capabilities but also amplified the computational demands for Approximate…
Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
Ruoyu Qin, Zheming Li, Weiran He +4
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. It features a KVCache-centric disaggregated architecture that separates the prefill and de…
Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
Shaoyuan Chen, Wencong Xiao, Yutong Lin +5
Transformer-based large language models (LLMs) exhibit impressive performance in generative tasks but also introduce significant challenges in real-world serving due to inefficient…
A Case for Asymmetric Non-Volatile Memory Architecture
Teng Ma, Mingxing Zhang, Kang Chen +2
The byte-addressable Non-Volatile Memory (NVM) is a promising technology since it simultaneously provides DRAM-like performance, disk-like capacity, and persistency. The current NV…
RFP: A Remote Fetching Paradigm for RDMA-Accelerated Systems
Maomeng Su, Mingxing Zhang, Kang Chen +2
Remote Direct Memory Access (RDMA) is an efficient way to improve the performance of traditional client-server systems. Currently, there are two main design paradigms for RDMA-acce…