16 citations · 18 across the 9 of their papers we have counts for
11 papers
EMA: Elastic and Performance Transparent Memory Across GPUs
Yi Xu, Tian Xia, Ion Stoica
Multi-GPU servers have become the standard building block of modern data centers, providing aggregated capacity through high-bandwidth interconnects. At the same time, workloads su…
mKernel: Fast Multi-GPU, Multi-Node Fused Kernels
Ziming Mao, Yihan Zhang, Shawn Wei Chew +5
Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granularity of kernels, on separate s…
CommBench: Can LLMs Write Correct and Efficient GPU Communication Code?
Shuang Ma, Yuyi Li, Yihan Zhang +12
Training and serving large language models (LLMs) rely heavily on high-performance GPU communication, yet implementing efficient GPU communication primitives requires deep expertis…
Flash-KMeans: Fast and Memory-Efficient Exact K-Means
Shuo Yang, Haocheng Xi, Yilong Zhao +10
-means has historically been positioned primarily as an offline processing primitive, typically used for dataset organization or embedding preprocessing rather than as a first-c…
SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing
Xuanyi Zhou, Qiuyang Mang, Shuo Yang +7
Diffusion Transformers (DiTs) have become a leading backbone for video generation, yet their quadratic attention cost remains a major bottleneck. Sparse attention reduces this cost…
UCCL-EP: Portable Expert-Parallel Communication
Ziming Mao, Yihan Zhang, Chihan Cui +9
Mixture-of-Experts (MoE) workloads rely on expert parallelism (EP) to achieve high GPU efficiency. State-of-the-art EP communication systems such as DeepEP demonstrate strong perfo…
SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost
Zhifei Li, Tian Xia, Ziming Mao +9
AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3-1…
SkyWalker: A Locality-Aware Cross-Region Load Balancer for LLM Inference
Tian Xia, Ziming Mao, Jamison Kerney +5
Serving Large Language Models (LLMs) efficiently in multi-region setups remains a challenge. Due to cost and GPU availability concerns, providers typically deploy LLMs in multiple…
An Extensible Software Transport Layer for GPU Networking
Yang Zhou, Zhongjie Chen, Ziming Mao +11
Fast-evolving machine learning (ML) workloads have increasing requirements for networking. However, host network transport on RDMA NICs is hard to evolve, causing problems for ML w…
Pie: Pooling CPU Memory for LLM Inference
Yi Xu, Ziming Mao, Xiangxi Mo +2
The rapid growth of LLMs has revolutionized natural language processing and AI analysis, but their increasing size and memory demands present significant challenges. A common solut…
SkyServe: Serving AI Models across Regions and Clouds with Spot Instances
Ziming Mao, Tian Xia, Zhanghao Wu +6
Recent years have witnessed an explosive growth of AI models. The high cost of hosting AI services on GPUs and their demanding service requirements, make it timely and challenging…