6 citations · 7 across the 12 of their papers we have counts for
7 papers · 1 filter
DecompRL: Solving Harder Problems by Learning Modular Code Generation
Juliette Decugis, Fabian Gloeckle, Francis Bach +2
How can Large Language Models (LLMs) solve problems they currently cannot? Repeated sampling scales test-time compute but GPU cost grows linearly with attempts, while reinforcement…
Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL
Juliette Decugis, Sean O'Brien, Francis Bach +2
Reinforcement learning post-training dramatically improves LLM reasoning, but suffers from training instability and diversity collapse. Advantage functions offer an appealing fix:…
Extrapolative Weight Averaging Reveals Correctness-Efficiency Frontiers in Code RL
Kunhao Zheng, Pierre Chambon, Juliette Decugis +4
Linear interpolation between fine-tuned checkpoints has been shown to trace the Pareto front between competing objectives, but whether extrapolative weight averaging can extend suc…
Efficient RL Training for LLMs with Experience Replay
Charles Arnal, Vivien Cabannes, Taco Cohen +2
While Experience Replay - the practice of storing rollouts and reusing them multiple times during training - is a foundational technique in general RL, it remains largely unexplore…
A Deep Dive into Scaling RL for Code Generation with Synthetic Data and Curricula
Cansu Sancaktar, David Zhang, Gabriel Synnaeve +1
Reinforcement learning (RL) has emerged as a powerful paradigm for improving large language models beyond supervised fine-tuning, yet sustaining performance gains at scale remains…
RL-finetuning LLMs from on- and off-policy data with a single algorithm
Yunhao Tang, Taco Cohen, David W. Zhang +2
We introduce a novel reinforcement learning algorithm (AGRO, for Any-Generation Reward Optimization) for fine-tuning large-language models. AGRO leverages the concept of generation…
Soft Policy Optimization: Online Off-Policy RL for Sequence Models
Taco Cohen, David W. Zhang, Kunhao Zheng +3
RL-based post-training of language models is almost exclusively done using on-policy methods such as PPO. These methods cannot learn from arbitrary sequences such as those produced…