4 papers · 1 filter
ReSPO: Reshaped Sequence Policy Optimization for Gradient Starvation in Off-Policy Learning
Yihang Chen, Yuanhao Ban, Cho-Jui Hsieh
Reinforcement learning from verifiable rewards (RLVR) frequently reuses rollouts across multiple policy updates, increasing the mismatch between the current policy and the data-gen…
ReCAST: Reward Credit Assignment across Timesteps for Online Diffusion Reinforcement
Yihang Chen, Yuanhao Ban, Kuei-Chun Kao +1
Training diffusion models with multiple rewards requires distinguishing user preference from reward informativeness. User preference determines how much each reward should contribu…
A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design
Tong Xie, Yuanhao Ban, Yunqi Hong +3
Supervised fine-tuning (SFT) typically maximizes the likelihood of every token in a demonstrated trajectory. However, an observed token can be non-unique, noisy, or misaligned with…
Rethinking RL Evaluation: Can Benchmarks Truly Reveal Failures of RL Methods?
Zihan Chen, Yiming Zhang, Hengguang Zhou +3
Current benchmarks are inadequate for evaluating progress in reinforcement learning (RL) for large language models (LLMs).Despite recent benchmark gains reported for RL, we find th…