9 papers
Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
Yicheng Feng, Xin Tan, Yangtao Deng +3
Modern LLM serving is no longer homogeneous or monolithic. Production systems now combine disaggregated execution, complex parallelism, runtime optimizations, and stateful workload…
ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
Kaiwen Chen, Xin Tan, Minchen Yu +2
Large Reasoning Models (LRMs) are becoming integral to many AI inference systems, enhancing their capabilities with advanced reasoning. However, deploying these models in productio…
GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injection
Chaobo Jia, Ruipeng Wan, Ting Sun +5
LLM-based game generation promises to turn natural-language specifications into executable games, but progress is limited by the lack of reliable automated verification. Unlike con…
Leveraging Large Language Models for Generalizing Peephole Optimizations
Chunhao Liao, Hongxu Xu, Xintong Zhou +2
Peephole optimizations are a core component of modern optimizing compilers. It rewrites specific instruction into semantically equivalent but more efficient forms. In practice, cre…
OrchestrRL: Dynamic Compute and Network Orchestration for Disaggregated RL
Xin Tan, Yicheng Feng, Yu Zhou +3
Disaggregating the generation and training stages in RL is widely adopted to scale LLM post-training. There are two critical challenges here. First, the generation stage often beco…
Make Every Draft Count: Hidden State based Speculative Decoding
Yuetao Chen, Xuliang Wang, Xinzhou Zheng +3
Speculative decoding has emerged as a pivotal technique to accelerate LLM inference by employing a lightweight draft model to generate candidate tokens that are subsequently verifi…