3 citations · 6 across the 14 of their papers we have counts for
9 papers · 1 filter
Ofan: Optimal Load Balancing for AI Training
Sarah McClure, Evyatar Cohen, Jakob Krebs +5
The extreme collective completion time (CCT) demands of AI workloads challenge existing packet spraying algorithms, which can have trouble efficiently load-balancing workloads that…
Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving
Dev Bali, Soujanya Ponnapalli, Yichuan Wang +3
LLM serving is typically offered as a shared, multi-tenant service, where high-demand workloads from one client can cause latency SLO violations for others. Existing solutions for…
mKernel: Fast Multi-GPU, Multi-Node Fused Kernels
Ziming Mao, Yihan Zhang, Shawn Wei Chew +5
Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granularity of kernels, on separate s…
On Topology's Role in ML Training Performance
Sarah McClure, Tegan Wilson, Brad Karp +4
Modern machine learning training workloads run on large-scale networks of compute accelerators. The networks commonly deployed in these systems are typically variations of two basi…
EnCoR: An end-to-end architecture for simplifying cellular networks
Wesley Woo, Zhuowei Wen, Monniiesh Velmurugan +4
Since their creation, cellular networks have made in-network mobility support a key feature of their service model. While this approach provides seamless connectivity for legacy tr…
Rethinking Network Topologies for Cost-Effective Mixture-of-Experts LLM Serving
Junsun Choi, Sam Son, Sunjin Choi +5
Mixture-of-experts (MoE) architectures have turned LLM serving into a cluster-scale workload in which communication consumes a considerable portion of LLM serving runtime. This has…
Delta Fair Sharing: Performance Isolation for Multi-Tenant Storage Systems
Tyler Griggs, Soujanya Ponnapalli, Dev Bali +8
Modern storage systems, often deployed to support multiple tenants in the cloud, must provide performance isolation. Unfortunately, traditional approaches such as fair sharing do n…
UCCL-EP: Portable Expert-Parallel Communication
Ziming Mao, Yihan Zhang, Chihan Cui +9
Mixture-of-Experts (MoE) workloads rely on expert parallelism (EP) to achieve high GPU efficiency. State-of-the-art EP communication systems such as DeepEP demonstrate strong perfo…
SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost
Zhifei Li, Tian Xia, Ziming Mao +9
AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3-1…