5 papers
Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
Shobhita Sundaram, John Quan, Ariel Kwiatkowski +3
RL methods for scaling large reasoning models stall on datasets with low initial success rates, and thus little training signal. We investigate a fundamental question: Can a pretra…
Likelihood-Based Reward Designs for General LLM Reasoning
Ariel Kwiatkowski, Natasha Butt, Ismail Labiad +2
Fine-tuning large language models (LLMs) on reasoning benchmarks via reinforcement learning requires a specific reward function, often binary, for each benchmark. This comes with t…
Soft Tokens, Hard Truths
Natasha Butt, Ariel Kwiatkowski, Ismail Labiad +2
The use of continuous instead of discrete tokens during the Chain-of-Thought (CoT) phase of reasoning LLMs has garnered attention recently, based on the intuition that a continuous…
Tackling the Zero-Shot Reinforcement Learning Loss Directly
Yann Ollivier
Zero-shot reinforcement learning (RL) methods aim at instantly producing a behavior for an RL task in a given environment, from a description of the reward function. These methods…
Which Features are Best for Successor Features?
Yann Ollivier
In reinforcement learning, universal successor features (SFs) are a way to provide zero-shot adaptation to new tasks at test time: they provide optimal policies for all downstream…