Preference as Reward, Maximum Preference Optimization with Importance Sampling

Zaifan Jiang, Xing Huang, Chao Wei

Preference learning is a key technology for aligning language models with human values. Reinforcement Learning from Human Feedback (RLHF) is a model-based algorithm to optimize pre…