6 papers
Router Drift Control: Mitigating Off-Policy Degradation in Mixture-of-Experts Reinforcement Learning
Liangyao Jin, Zhixin Wang, Liming Liu +8
Reinforcement learning (RL) can become off-policy due to training-inference mismatch, mini-batch updates, and asynchronous training. The resulting mismatch is especially harmful in…
VenusRL: A Fully Disaggregated Agentic RL System with Priority Scheduling and Scalable Interaction
Mingjun Zhang, Yucheng Li, Menghao Zhang +15
Agentic Reinforcement Learning (RL) trains LLM agents through multi-turn interactions with external tool environments. Its multi-turn nature exposes two system-level bottlenecks un…
Demystifying the Design Space and Best Practices for Heterogeneous LLM Inference and Serving
Zhixin Wang, Zhengbo Wang, Fangcheng Fu +15
Heterogeneous prefill-decode (PD) inference is now in production: prefill on cost-efficient or supply-available accelerators, decode on bandwidth-strong ones, and KV state crossing…
PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
Yiqi Zhang, Fangzheng Jiao, Tian Tang +13
Reinforcement learning with verifiable rewards (RLVR) has recently unlocked strong reasoning capabilities in large language models (LLMs), triggering rapid exploration of new algor…
Memory Analysis on the Training Course of DeepSeek Models
Ping Zhang, Lei Su
We present a theoretical analysis of GPU memory consumption during the training of DeepSeek models such as DeepSeek-v2 and DeepSeek-v3. Our primary objective is to clarify the devi…
Topology-aware Preemptive Scheduling for Co-located LLM Workloads
Ping Zhang, Lei Su, Jinjie Yang +1
Hosting diverse large language model workloads in a unified resource pool through co-location is cost-effective. For example, long-running chat services generally follow diurnal tr…