5 papers
LiveR: Fine-Grained Elasticity via Live Reconfiguration for Model Training
Haoyuan Liu, Kairui Zhou, Shuyao Qi +4
To reduce user costs and maximize cluster utilization, large model training increasingly leverages volatile but inexpensive GPU capacity, such as spot instances and reclaimable res…
Libra: Accelerating Socket I/O via Programmable Selective Data Copying
Kairui Zhou, Shengkai Lin, Wei Zhang +1
Layer-7 (L7) proxies are critical to modern cloud-native systems, yet their performance is increasingly bottlenecked by copying entire payloads across the kernel-user boundary. Exi…
FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training
Shuyao Qi, Haoyuan Liu, Shizhen Zhao
Fine-grained, per-micro-batch load balancing is essential for efficient Mixture-of-Experts (MoE) training, yet every prior dynamic scheduling scheme pays for it with extra communic…
SAIR: Cost-Efficient Multi-Stage ML Pipeline Autoscaling via In-Context Reinforcement Learning
Jianchang Su, Yifan Zhang, Shengkai Lin +4
Multi-stage ML inference pipelines are difficult to autoscale due to heterogeneous resources, cross-stage coupling, and dynamic bottleneck migration. We present SAIR, an autoscalin…
SHIFT: Exploring the Boundary of RDMA Network Fault Tolerance
Shengkai Lin, Kairui Zhou, Hongtao Zhang +7
Under gang scheduling for large-scale distributed large language model (LLM) training, a single network anomaly can stall or abort an entire job. Current network fault tolerance me…