S2T-RLHF: Hierarchical Credit Assignment for Stable Preference-Based RLHF

Wei Chen, Guanghui Zhu, Yafei Li +2

Reinforcement learning from human feedback (RLHF) with preference-based reward models often exhibits unstable training dynamics. A key contributing factor is that standard RLHF rel…