7 papers
Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
Yicheng Feng, Xin Tan, Yangtao Deng +3
Modern LLM serving is no longer homogeneous or monolithic. Production systems now combine disaggregated execution, complex parallelism, runtime optimizations, and stateful workload…
OrchestrRL: Dynamic Compute and Network Orchestration for Disaggregated RL
Xin Tan, Yicheng Feng, Yu Zhou +3
Disaggregating the generation and training stages in RL is widely adopted to scale LLM post-training. There are two critical challenges here. First, the generation stage often beco…
DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
Zili Zhang, Yinmin Zhong, Yimin Jiang +6
Multimodal large language models (LLMs) empower LLMs to ingest inputs and generate outputs in multiple forms, such as text, image, and audio. However, the integration of multiple m…
Frontier: Simulating the Next Generation of LLM Inference Systems
Yicheng Feng, Xin Tan, Kin Hang Sew +3
Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like…
InfiniteHBD: Building Datacenter-Scale High-Bandwidth Domain for LLM with Optical Circuit Switching Transceivers
Chenchen Shou, Guyue Liu, Hao Nie +11
Scaling Large Language Model (LLM) training relies on multi-dimensional parallelism, where High-Bandwidth Domains (HBDs) are critical for communication-intensive parallelism like T…
Teola: Towards End-to-End Optimization of LLM-based Applications
Xin Tan, Yimin Jiang, Yitao Yang +1
Large language model (LLM)-based applications consist of both LLM and non-LLM components, each contributing to the end-to-end latency. Despite great efforts to optimize LLM inferen…