Critique-out-Loud Reward Models

Zachary Ankner, Mansheej Paul, Brandon Cui +2

Traditionally, reward models used for reinforcement learning from human feedback (RLHF) are trained to directly predict preference scores without leveraging the generation capabili…