12 papers
AsyncWebRL: Efficient Asynchronous Reinforcement Learning for Multi-Step Visual Web Agents
Hao Bai, Rui Yang, Chenlu Ye +3
Training vision-language web agents with multi-step RL is compute-intensive, with two dominant forms of inefficiency: idle GPUs in synchronous RL, and trajectories that use more st…
Adaptive Layerwise Perturbation: Unifying Off-Policy Corrections for LLM RL
Chenlu Ye, Xuanchang Zhang, Yifan Hao +6
Off-policy problems such as policy staleness and training--inference mismatch have become a major bottleneck for training stability and further exploration in LLM RL. The distribut…
Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training
Chenlu Ye, Zhou Yu, Ziji Zhang +5
Reinforcement Learning with Verifiable Rewards (RLVR) improves final-answer accuracy on reasoning tasks, but it does not reliably improve reasoning quality. Because outcome rewards…
Rethinking Importance Sampling in LLM Policy Optimization: A Cumulative Token Perspective
Yuheng Zhang, Chenlu Ye, Shuowei Jin +4
Reinforcement learning, including reinforcement learning with verifiable rewards (RLVR), has emerged as a powerful approach for LLM post-training. Central to these approaches is th…
Logarithmic Regret for Online KL-Regularized Reinforcement Learning
Heyang Zhao, Chenlu Ye, Wei Xiong +2
Recent advances in Reinforcement Learning from Human Feedback (RLHF) have shown that KL-regularization plays a pivotal role in improving the efficiency of RL fine-tuning for large…
Reinforce-Ada: An Adaptive Sampling Framework under Non-linear RL Objectives
Wei Xiong, Chenlu Ye, Baohao Liao +6
Reinforcement learning (RL) for large language model reasoning is frequently hindered by signal loss, a phenomenon where standard uniform sampling with small group sizes fails to u…