7 papers
A Few GPUs, A Whole Lotta Scale: Faithful LLM Training Emulation with PrismLLM
Shaoke Xi, ChonLam Lao, Boyi Jia +11
Large language model (LLM) training today runs on clusters spanning thousands of GPUs. While this scale enables rapid model advances, developing, debugging, and performance-tuning…
TrainMover: An Interruption-Resilient Runtime for ML Training
ChonLam Lao, Jiaqi Gao, Jiamin Cao +13
Large-scale ML training jobs are frequently interrupted by hardware and software anomalies, failures, and management events. Existing solutions like checkpoint-restart or runtime r…
Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving
Shan Yu, Junyi Shu, Yuanjiang Ni +14
As LLM applications grow more complex, developers are increasingly adopting multi-agent architectures to decompose workflows into specialized, collaborative components, introducing…
ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
Yuxing Xiang, Xue Li, Kun Qian +3
With the widespread adoption of Large Language Models (LLMs), serving LLM inference requests has become an increasingly important task, attracting active research advancements. Pra…
EROICA: Online Performance Troubleshooting for Large-scale Model Training
Yu Guan, Zhiyu Yin, Haoyu Chen +11
Troubleshooting performance problems of large model training (LMT) is immensely challenging, due to unprecedented scales of modern GPU clusters, the complexity of software-hardware…
Model-Based Diagnosis: Automating End-to-End Diagnosis of Network Failures
Changrong Wu, Yiyao Yu, Myungjin Lee +4
Fast diagnosis and repair of enterprise network failures is critically important since disruptions cause major business impacts. Prior works focused on diagnosis primitives or proc…