Mitigating Length Bias in RLHF through a Causal Lens

Hyeonji Kim, Sujeong Oh, Sanghack Lee

Reinforcement learning from human feedback (RLHF) is widely used to align large language models (LLMs) with human preferences. However, RLHF-trained reward models often exhibit len…