4 papers · 1 filter
Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweighting
Yunzhen Feng, Parag Jain, Anthony Hartshorn +2
Reinforcement learning with verifiable rewards (RLVR) has become a standard recipe for improving large language models (LLMs) on reasoning tasks, with Group Relative Policy Optimiz…
On the optimization dynamics of RLVR: Gradient gap and step size thresholds
Joe Suk, Yaqi Duan
Reinforcement Learning with Verifiable Rewards (RLVR), which uses simple binary feedback to post-train large language models, has found significant empirical success. However, a pr…
PILAF: Optimal Human Preference Sampling for Reward Modeling
Yunzhen Feng, Ariel Kwiatkowski, Kunhao Zheng +2
As large language models increasingly drive real-world applications, aligning them with human values becomes paramount. Reinforcement Learning from Human Feedback (RLHF) has emerge…
Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces
Yaqi Duan, Martin J. Wainwright
We introduce a novel framework for analyzing reinforcement learning (RL) in continuous state-action spaces, and use it to prove fast rates of convergence in both off-line and on-li…