6 papers
LiveR: Fine-Grained Elasticity via Live Reconfiguration for Model Training
Haoyuan Liu, Kairui Zhou, Shuyao Qi +4
To reduce user costs and maximize cluster utilization, large model training increasingly leverages volatile but inexpensive GPU capacity, such as spot instances and reclaimable res…
DynaFlow: Transparent and Flexible Intra-Device Parallelism via Programmable Operator Scheduling
Yi Pan, Yile Gu, Jinbin Luo +7
Intra-device parallelism addresses resource under-utilization in ML inference and training by overlapping the execution of operators with different resource usage. However, its wid…
Libra: Accelerating Socket I/O via Programmable Selective Data Copying
Kairui Zhou, Shengkai Lin, Wei Zhang +1
Layer-7 (L7) proxies are critical to modern cloud-native systems, yet their performance is increasingly bottlenecked by copying entire payloads across the kernel-user boundary. Exi…
SHIFT: Exploring the Boundary of RDMA Network Fault Tolerance
Shengkai Lin, Kairui Zhou, Hongtao Zhang +7
Under gang scheduling for large-scale distributed large language model (LLM) training, a single network anomaly can stall or abort an entire job. Current network fault tolerance me…
SAIR: Cost-Efficient Multi-Stage ML Pipeline Autoscaling via In-Context Reinforcement Learning
Jianchang Su, Yifan Zhang, Shengkai Lin +4
Multi-stage ML inference pipelines are difficult to autoscale due to heterogeneous resources, cross-stage coupling, and dynamic bottleneck migration. We present SAIR, an autoscalin…
A Highly Scalable LLM Clusters with Optical Interconnect
Xinchi Han, Yongxi Lv, Weihao Jiang +8
Recent years have witnessed the adoption of optical circuit switch (OCS) technology. How to design the physical topology, defined by the physical wiring between electrical switchin…