6 papers · 1 filter
VenusRL: A Fully Disaggregated Agentic RL System with Priority Scheduling and Scalable Interaction
Mingjun Zhang, Yucheng Li, Menghao Zhang +15
Agentic Reinforcement Learning (RL) trains LLM agents through multi-turn interactions with external tool environments. Its multi-turn nature exposes two system-level bottlenecks un…
Efficient Agentic LLM Serving over SSD-based Sparse KV Storage
Wenhao He, Ping Zhang, Xiaohe Hu +5
Agentic sessions driven by Large language models (LLMs) often alternate between model inference and tool use, accumulating long histories across successive rounds. Serving these se…
DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale
Jialiang Huang, Hongxuan Tang, Jingchang Chen +128
Large-scale agentic training and evaluation with large language models (LLMs) rely on isolated, stateful execution environments in which models inspect repositories, invoke tools,…
DualDecoder: Accelerate Long Context LLM Inference by Predictive Prefetch
Zuning Liang, Zhiyi Yao, Qi Chen +6
Long-context inference is becoming a fundamental capability for modern LLM serving, especially driven by emerging agentic applications. Yet it faces a severe memory wall that the K…
Demystifying the Design Space and Best Practices for Heterogeneous LLM Inference and Serving
Zhixin Wang, Zhengbo Wang, Fangcheng Fu +15
Heterogeneous prefill-decode (PD) inference is now in production: prefill on cost-efficient or supply-available accelerators, decode on bandwidth-strong ones, and KV state crossing…
An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters
Mingjun Zhang, Xiaohe Hu, Menghao Zhang +21
Large-scale LLM training requires collective communication libraries to exchange data among distributed GPUs. As a company dedicated to building and operating large-scale GPU train…