Score Centering Stabilizes Off-policy Reinforcement Learning

Martin Marek, Max Ryabinin

Reinforcement learning (RL) of large language models is notoriously sensitive to small differences between training and inference engines, often referred to as the training-inferen…