5 papers
A Few GPUs, A Whole Lotta Scale: Faithful LLM Training Emulation with PrismLLM
Shaoke Xi, ChonLam Lao, Boyi Jia +11
Large language model (LLM) training today runs on clusters spanning thousands of GPUs. While this scale enables rapid model advances, developing, debugging, and performance-tuning…
MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inference
Abhimanyu Rajeshkumar Bambhaniya, Hanjiang Wu, Suvinay Subramanian +8
Modern LLM serving now spans multi-stage pipelines including RAG retrieval and KV cache reuse, each with distinct compute, memory, and latency demands. Inference engines expose a l…
Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
Weiqi Feng, Yangrui Chen, Shaoyu Wang +3
Multimodal large language models (MLLMs) have extended the success of large language models (LLMs) to multiple data types, such as image, text and audio, achieving significant perf…
Towards Easy and Realistic Network Infrastructure Testing for Large-scale Machine Learning
Jinsun Yoo, ChonLam Lao, Lianjie Cao +4
This paper lays the foundation for Genie, a testing framework that captures the impact of real hardware network behavior on ML workload performance, without requiring expensive GPU…
Minder: Faulty Machine Detection for Large-scale Distributed Model Training
Yangtao Deng, Xiang Shi, Zhuo Jiang +12
Large-scale distributed model training requires simultaneous training on up to thousands of machines. Faulty machine detection is critical when an unexpected fault occurs in a mach…