13 papers
CommBench: Can LLMs Write Correct and Efficient GPU Communication Code?
Shuang Ma, Yuyi Li, Yihan Zhang +12
Training and serving large language models (LLMs) rely heavily on high-performance GPU communication, yet implementing efficient GPU communication primitives requires deep expertis…
The Time is Here for Just-in-Time Systems: Challenges and Opportunities
Shu Liu, Alexander Krentsel, Shubham Agarwal +8
Core systems like key-value stores have historically taken years to build, and are designed to be general so as to amortize cost across deployments, paying a significant performanc…
UCCL-Zip: Lossless Compression Supercharged GPU Communication
Shuang Ma, Chon Lam Lao, Zhiying Xu +8
The rapid growth of large language models (LLMs) has made GPU communication a critical bottleneck. While prior work reduces communication volume via quantization or lossy compressi…
K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
Shiyi Cao, Ziming Mao, Joseph E. Gonzalez +1
Optimizing GPU kernels is critical for efficient modern machine learning systems yet remains challenging due to the complex interplay of design factors and rapid hardware evolution…
UCCL-EP: Portable Expert-Parallel Communication
Ziming Mao, Yihan Zhang, Chihan Cui +9
Mixture-of-Experts (MoE) workloads rely on expert parallelism (EP) to achieve high GPU efficiency. State-of-the-art EP communication systems such as DeepEP demonstrate strong perfo…
SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost
Zhifei Li, Tian Xia, Ziming Mao +9
AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3-1…