Towards Efficient Online Exploration for Reinforcement Learning with Human Feedback

Gen Li, Yuling Yan

Reinforcement learning with human feedback (RLHF), which learns a reward model from human preference data and then optimizes a policy to favor preferred responses, has emerged as a…