1 citations · 2 across the 11 of their papers we have counts for
11 papers
Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
Yicheng Feng, Xin Tan, Yangtao Deng +3
Modern LLM serving is no longer homogeneous or monolithic. Production systems now combine disaggregated execution, complex parallelism, runtime optimizations, and stateful workload…
Libra: Efficient Resource Management for Agentic RL Post-Training
Kaiwen Chen, Xin Tan, Jingzong Li +6
Reinforcement learning (RL) has emerged as a standard post-training paradigm for shaping large language models (LLMs) into capable agents. In agentic RL, the rollout stage generate…
GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injection
Chaobo Jia, Ruipeng Wan, Ting Sun +5
LLM-based game generation promises to turn natural-language specifications into executable games, but progress is limited by the lack of reliable automated verification. Unlike con…
Leveraging Large Language Models for Generalizing Peephole Optimizations
Chunhao Liao, Hongxu Xu, Xintong Zhou +2
Peephole optimizations are a core component of modern optimizing compilers. It rewrites specific instruction into semantically equivalent but more efficient forms. In practice, cre…
Make Every Draft Count: Hidden State based Speculative Decoding
Yuetao Chen, Xuliang Wang, Xinzhou Zheng +3
Speculative decoding has emerged as a pivotal technique to accelerate LLM inference by employing a lightweight draft model to generate candidate tokens that are subsequently verifi…
OrchestrRL: Dynamic Compute and Network Orchestration for Disaggregated RL
Xin Tan, Yicheng Feng, Yu Zhou +3
Disaggregating the generation and training stages in RL is widely adopted to scale LLM post-training. There are two critical challenges here. First, the generation stage often beco…
Mycroft: Tracing Dependencies in Collective Communication Towards Reliable LLM Training
Yangtao Deng, Lei Zhang, Qinlong Wang +13
Reliability is essential for ensuring efficiency in LLM training. However, many real-world reliability issues remain difficult to resolve, resulting in wasted resources and degrade…
Frontier: Simulating the Next Generation of LLM Inference Systems
Yicheng Feng, Xin Tan, Kin Hang Sew +3
Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like…
ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
Kaiwen Chen, Xin Tan, Minchen Yu +2
Large Reasoning Models (LRMs) are becoming integral to many AI inference systems, enhancing their capabilities with advanced reasoning. However, deploying these models in productio…
DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Training
Xin Tan, Yuetao Chen, Yimin Jiang +6
Diffusion Transformers (DiTs) have shown remarkable performance in generating high-quality videos. However, the quadratic complexity of 3D full attention remains a bottleneck in sc…
Echo: Simulating Distributed Training At Scale
Yicheng Feng, Yuetao Chen, Kaiwen Chen +7
Simulation offers unique values for both enumeration and extrapolation purposes, and is becoming increasingly important for managing the massive machine learning (ML) clusters and…