4 papers
Multi-stage Flow Scheduling for LLM Serving
Yijun Sun, Xudong Liao, Songrun Xie +5
Meeting stringent Time-To-First-Token (TTFT) requirements is crucial for LLM applications. To improve efficiency, modern LLM serving systems adopt disaggregated architectures with…
Analyzing Communication Predictability in LLM Training
Wenxue Li, Xiangzhou Liu, Yuxuan Li +9
Effective communication is essential in distributed training, with predictability being one of its most significant characteristics. However, existing studies primarily focus on ex…
MixNet: A Runtime Reconfigurable Optical-Electrical Fabric for Distributed Mixture-of-Experts Training
Xudong Liao, Yijun Sun, Han Tian +13
Mixture-of-Expert (MoE) models outperform conventional models by selectively activating different subnets, named experts, on a per-token basis. This gated computation generates dyn…
Design and Operation of Shared Machine Learning Clusters on Campus
Kaiqiang Xu, Decang Sun, Hao Wang +6
Amid the rapid advancements in large machine learning (ML) models, universities worldwide are investing substantial funds and efforts into GPU clusters. However, managing a shared…