4 papers
Libra: Accelerating Socket I/O via Programmable Selective Data Copying
Kairui Zhou, Shengkai Lin, Wei Zhang +1
Layer-7 (L7) proxies are critical to modern cloud-native systems, yet their performance is increasingly bottlenecked by copying entire payloads across the kernel-user boundary. Exi…
SAIR: Cost-Efficient Multi-Stage ML Pipeline Autoscaling via In-Context Reinforcement Learning
Jianchang Su, Yifan Zhang, Shengkai Lin +4
Multi-stage ML inference pipelines are difficult to autoscale due to heterogeneous resources, cross-stage coupling, and dynamic bottleneck migration. We present SAIR, an autoscalin…
SHIFT: Exploring the Boundary of RDMA Network Fault Tolerance
Shengkai Lin, Kairui Zhou, Hongtao Zhang +7
Under gang scheduling for large-scale distributed large language model (LLM) training, a single network anomaly can stall or abort an entire job. Current network fault tolerance me…
A Highly Scalable LLM Clusters with Optical Interconnect
Xinchi Han, Yongxi Lv, Weihao Jiang +8
Recent years have witnessed the adoption of optical circuit switch (OCS) technology. How to design the physical topology, defined by the physical wiring between electrical switchin…