4 papers
Multi-stage Flow Scheduling for LLM Serving
Yijun Sun, Xudong Liao, Songrun Xie +5
Meeting stringent Time-To-First-Token (TTFT) requirements is crucial for LLM applications. To improve efficiency, modern LLM serving systems adopt disaggregated architectures with…
Analyzing Communication Predictability in LLM Training
Wenxue Li, Xiangzhou Liu, Yuxuan Li +9
Effective communication is essential in distributed training, with predictability being one of its most significant characteristics. However, existing studies primarily focus on ex…
MixNet: A Runtime Reconfigurable Optical-Electrical Fabric for Distributed Mixture-of-Experts Training
Xudong Liao, Yijun Sun, Han Tian +13
Mixture-of-Expert (MoE) models outperform conventional models by selectively activating different subnets, named experts, on a per-token basis. This gated computation generates dyn…
Swift: Rethinking RDMA Control Plane for Elastic Computing
Junxue Zhang, Han Tian, Xinyang Huang +5
Elastic computing enables dynamic scaling to meet workload demands, and Remote Direct Memory Access (RDMA) enhances this by providing high-throughput, low-latency network communica…