5 papers
Staleness-Learning Rate Scaling Laws for Asynchronous RLHF
Jingwei Song, Haofeng Xu, Jie Xiao +8
High-throughput RLHF systems often decouple rollout generation from policy optimization, leading to the use of stale rollouts during learner updates. In this work, we study the eff…
ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning
Jingwei Song, Meng Chen, Jie Xiao +15
Reinforcement learning (RL) is a critical stage in post-training large language models (LLMs), involving repeated interaction between rollout generation, reward evaluation, and cen…
MARS: Unleashing the Power of Speculative Decoding via Margin-Aware Verification
Jingwei Song, Xinyu Wang, Hanbin Wang +6
Speculative Decoding (SD) accelerates autoregressive large language model (LLM) inference by decoupling generation and verification. While recent methods improve draft quality by t…
Co-Evolution of Policy and Internal Reward for Language Agents
Xinyu Wang, Hanwei Wu, Jingwei Song +8
Large language model (LLM) agents learn by interacting with environments, but long-horizon training remains fundamentally bottlenecked by sparse and delayed rewards. Existing metho…
Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput
Jingwei Song, Wanyi Chen, Xinyuan Song +7
Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens that are later verified by a stronger target model. While…