Learning a Pessimistic Reward Model in RLHF

Yinglun Xu, Hangoo Kang, Tarun Suresh +2

This work proposes `PET', a novel pessimistic reward fine-tuning method, to learn a pessimistic reward model robust against reward hacking in offline reinforcement learning from hu…