5 papers
Message Passing Enables Efficient Reasoning
Xuecheng Liu, Daman Arora, Gokul Swamy +1
While inference-time scaling has improved the reasoning abilities of large language models (LLMs), the need to generate long chains-of-thought (CoTs) is a computational bottleneck.…
SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learning
Ruiqi Zhang, Daman Arora, Song Mei +1
Training large language models with reinforcement learning (RL) against verifiable rewards significantly enhances their reasoning abilities, yet remains computationally expensive d…
Shrinking the Variance: Shrinkage Baselines for Reinforcement Learning with Verifiable Rewards
Guanning Zeng, Zhaoyi Zhou, Daman Arora +1
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful paradigm for post-training large reasoning models (LRMs) using policy-gradient methods such as GRPO.…
Maximum Likelihood Reinforcement Learning
Fahim Tajwar, Guanning Zeng, Yueer Zhou +7
Reinforcement learning (RL) is the method of choice for training models in setups where the objective function can only be evaluated by sampling from the model. Our key observation…
Training Language Models to Reason Efficiently
Daman Arora, Andrea Zanette
Scaling model size and training data has led to great advances in the performance of Large Language Models (LLMs). However, the diminishing returns of this approach necessitate alt…