3 citations · 6 across the 14 of their papers we have counts for
17 papers
Ofan: Optimal Load Balancing for AI Training
Sarah McClure, Evyatar Cohen, Jakob Krebs +5
The extreme collective completion time (CCT) demands of AI workloads challenge existing packet spraying algorithms, which can have trouble efficiently load-balancing workloads that…
Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving
Dev Bali, Soujanya Ponnapalli, Yichuan Wang +3
LLM serving is typically offered as a shared, multi-tenant service, where high-demand workloads from one client can cause latency SLO violations for others. Existing solutions for…
mKernel: Fast Multi-GPU, Multi-Node Fused Kernels
Ziming Mao, Yihan Zhang, Shawn Wei Chew +5
Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granularity of kernels, on separate s…
On Topology's Role in ML Training Performance
Sarah McClure, Tegan Wilson, Brad Karp +4
Modern machine learning training workloads run on large-scale networks of compute accelerators. The networks commonly deployed in these systems are typically variations of two basi…
EnCoR: An end-to-end architecture for simplifying cellular networks
Wesley Woo, Zhuowei Wen, Monniiesh Velmurugan +4
Since their creation, cellular networks have made in-network mobility support a key feature of their service model. While this approach provides seamless connectivity for legacy tr…
Rethinking Network Topologies for Cost-Effective Mixture-of-Experts LLM Serving
Junsun Choi, Sam Son, Sunjin Choi +5
Mixture-of-experts (MoE) architectures have turned LLM serving into a cluster-scale workload in which communication consumes a considerable portion of LLM serving runtime. This has…
Delta Fair Sharing: Performance Isolation for Multi-Tenant Storage Systems
Tyler Griggs, Soujanya Ponnapalli, Dev Bali +8
Modern storage systems, often deployed to support multiple tenants in the cloud, must provide performance isolation. Unfortunately, traditional approaches such as fair sharing do n…
UCCL-EP: Portable Expert-Parallel Communication
Ziming Mao, Yihan Zhang, Chihan Cui +9
Mixture-of-Experts (MoE) workloads rely on expert parallelism (EP) to achieve high GPU efficiency. State-of-the-art EP communication systems such as DeepEP demonstrate strong perfo…
SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost
Zhifei Li, Tian Xia, Ziming Mao +9
AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3-1…
Load Balancing for AI Training Workloads
Sarah McClure, Evyatar Cohen, Alex Shpiner +4
The extreme bandwidth demands of AI training has made load-balancing a critical component in AI fabrics, and a variety of load-balancing designs have emerged in recent work from bo…
SkyWalker: A Locality-Aware Cross-Region Load Balancer for LLM Inference
Tian Xia, Ziming Mao, Jamison Kerney +5
Serving Large Language Models (LLMs) efficiently in multi-region setups remains a challenge. Due to cost and GPU availability concerns, providers typically deploy LLMs in multiple…
TURBO: Utility-Aware Bandwidth Allocation for Cloud-Augmented Autonomous Control
Peter Schafhalter, Alexander Krentsel, Hongbo Wei +4
Autonomous driving system progress has been driven by improvements in machine learning models, whose computational demands now exceed what edge devices alone can provide. The cloud…
Revisiting Cache Freshness for Emerging Real-Time Applications
Ziming Mao, Rishabh Iyer, Scott Shenker +1
Caching is widely used in industry to improve application performance by reducing data-access latency and taking the load off the backend infrastructure. TTLs have become the de-fa…
A Decidable Case of Query Determinacy: Project-Select Views
Wen Zhang, Aurojit Panda, Mooly Sagiv +1
Query determinacy is decidable for project-select views and a project-select-join query with no self joins, as long as the selection predicates are in a first-order theory for whic…
Extracting Database Access-Control Policies From Web Applications
Wen Zhang, Dev Bali, Jamison Kerney +2
To safeguard sensitive user data, web developers typically rely on implicit access-control policies, which they implement using access checks and query filters. This ad hoc approac…
Managing Bandwidth: The Key to Cloud-Assisted Autonomous Driving
Alexander Krentsel, Peter Schafhalter, Joseph E. Gonzalez +3
Prevailing wisdom asserts that one cannot rely on the cloud for critical real-time control systems like self-driving cars. We argue that we can, and must. Following the trends of i…
Blockaid: Data Access Policy Enforcement for Web Applications
Wen Zhang, Eric Sheng, Michael Chang +3
Modern web applications serve large amounts of sensitive user data, access to which is typically governed by data-access policies. Enforcing such policies is crucial to preventing…