Online Bandit Learning with Offline Preference Data for Improved RLHF

Akhil Agnihotri, Rahul Jain, Deepak Ramachandran +1

Reinforcement Learning with Human Feedback (RLHF) is at the core of fine-tuning methods for generative AI models for language and images. Such feedback is often sought as rank or p…