7 papers
PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
Yiqi Zhang, Fangzheng Jiao, Tian Tang +13
Reinforcement learning with verifiable rewards (RLVR) has recently unlocked strong reasoning capabilities in large language models (LLMs), triggering rapid exploration of new algor…
SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding
Shenggui Li, Chao Wang, Yikai Zhu +14
Large language models incur high inference latency due to sequential autoregressive decoding. Speculative decoding alleviates this bottleneck by using a lightweight draft model to…
CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control
Qiaoling Chen, Zhisheng Ye, Tian Tang +7
Batch inference for agentic workloads stresses the GPU key-value (KV) cache in a sustained and cumulative manner, often causing severe throughput degradation well before memory cap…
Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
AGI Team, Yuxuan Cai, Lu Chen +62
The evolution of Large Language Models (LLMs) from passive responders to autonomous agents necessitates a fundamental shift in learning paradigms -- from static imitation to incent…
ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems
Qiaoling Chen, Zijun Liu, Peng Sun +6
Adapting large language models (LLMs) via reinforcement learning (RL) is often bottlenecked by the generation stage, which can consume over 75\% of the training time. Speculative d…
Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
Ziming Liu, Boyu Tian, Guoteng Wang +15
Mixture-of-Experts (MoE) models challenge serving infrastructures with dynamic, sparse expert utilization, causing instability on conventional systems designed for dense architectu…