Representation-Aware Advantage Estimation: Your Reward Model Provides More Than A Scalar Output

Guozheng Li, Xiyan Fu, Yiwen Guo

Current reinforcement learning from human feedback (RLHF) methods primarily rely on scalar rewards from a trained reward model (RM). While effective, scalar rewards are often noisy…