4 papers
Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
Yicheng Feng, Xin Tan, Yangtao Deng +3
Modern LLM serving is no longer homogeneous or monolithic. Production systems now combine disaggregated execution, complex parallelism, runtime optimizations, and stateful workload…
OrchestrRL: Dynamic Compute and Network Orchestration for Disaggregated RL
Xin Tan, Yicheng Feng, Yu Zhou +3
Disaggregating the generation and training stages in RL is widely adopted to scale LLM post-training. There are two critical challenges here. First, the generation stage often beco…
Frontier: Simulating the Next Generation of LLM Inference Systems
Yicheng Feng, Xin Tan, Kin Hang Sew +3
Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like…
Echo: Simulating Distributed Training At Scale
Yicheng Feng, Yuetao Chen, Kaiwen Chen +7
Simulation offers unique values for both enumeration and extrapolation purposes, and is becoming increasingly important for managing the massive machine learning (ML) clusters and…