Detecting Prefix Bias in LLM-based Reward Models

Ashwin Kumar, Yuzi He, Aram H. Markosyan +2

Reinforcement Learning with Human Feedback (RLHF) has emerged as a key paradigm for task-specific fine-tuning of language models using human preference data. While numerous publicl…