4 papers
Democratizing MoE inference on commodity GPUs with CoMoE
Ruwen Fan, Yuezhi Zu, Minhui Xie +6
Deploying Mixture-of-Experts (MoE) models relies heavily on Expert Parallelism, which generates intense inter-GPU communication. Consequently, state-of-the-art inference systems re…
SAC: Disaggregated KV Cache System for Sparse Attention LLMs with CXL
Ruiyang Ma, Teng Ma, Junru Li +7
The scaling of LLMs toward long-context inference has shifted the primary serving system bottleneck from computation to memory capacity. Traditional solutions for dense attention m…
Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
Xinjun Yang, Qingda Hu, Junru Li +9
The rapid increase in LLM model sizes and the growing demand for long-context inference have made memory a critical bottleneck in GPU-accelerated serving systems. Although high-ban…
PolarStore: High-Performance Data Compression for Large-Scale Cloud-Native Databases
Qingda Hu, Xinjun Yang, Feifei Li +9
In recent years, resource elasticity and cost optimization have become essential for RDBMSs. While cloud-native RDBMSs provide elastic computing resources via disaggregated computi…