5 papers
Can LLMs Anticipate Their Cruxes?
David Reber, James Reber, Ari Holtzman +1
To use language models to help us make decisions we would like to understand the rationale behind the language model's outputs. In principle, the language model provides its ration…
Let it Calm: Exploratory Annealed Decoding for Verifiable Reinforcement Learning
Chenghao Yang, Lin Gui, Chenxiao Yang +3
Reinforcement learning with verifiable rewards (RLVR) is a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs), yet its success hinges on eff…
Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Training
Junkai Zhang, Zihao Wang, Lin Gui +7
Reinforcement fine-tuning (RFT) often suffers from reward over-optimization, where a policy model hacks the reward signals to achieve high scores while producing low-quality output…
RATE: Causal Explainability of Reward Models with Imperfect Counterfactuals
David Reber, Sean Richardson, Todd Nief +2
Reward models are widely used as proxies for human preferences when aligning or evaluating LLMs. However, reward models are black boxes, and it is often unclear what, exactly, they…
BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Sampling
Lin Gui, Cristina Gârbacea, Victor Veitch
This paper concerns the problem of aligning samples from large language models to human preferences using best-of- sampling, where we draw samples, rank them, and return the…