4 papers
SAC: Disaggregated KV Cache System for Sparse Attention LLMs with CXL
Ruiyang Ma, Teng Ma, Junru Li +7
The scaling of LLMs toward long-context inference has shifted the primary serving system bottleneck from computation to memory capacity. Traditional solutions for dense attention m…
Faster Gradient Methods for Highly-Smooth Stochastic Bilevel Optimization
Lesi Chen, Junru Li, El Mahdi Chayti +1
This paper studies the complexity of finding an -stationary point for stochastic bilevel optimization when the upper-level problem is nonconvex and the lower-level problem is s…
Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
Xinjun Yang, Qingda Hu, Junru Li +9
The rapid increase in LLM model sizes and the growing demand for long-context inference have made memory a critical bottleneck in GPU-accelerated serving systems. Although high-ban…
PolarStore: High-Performance Data Compression for Large-Scale Cloud-Native Databases
Qingda Hu, Xinjun Yang, Feifei Li +9
In recent years, resource elasticity and cost optimization have become essential for RDBMSs. While cloud-native RDBMSs provide elastic computing resources via disaggregated computi…