Reward Learning From Preference With Ties

Jinsong Liu, Dongdong Ge, Ruihao Zhu

Reward learning plays a pivotal role in Reinforcement Learning from Human Feedback (RLHF), ensuring the alignment of language models. The Bradley-Terry (BT) model stands as the pre…