Mitigating Cognitive Bias in RLHF by Altering Rationality

Tiffany Horter, Andrew Markham, Niki Trigoni +1

How can we make models robust to even imperfect human feedback? In reinforcement learning from human feedback (RLHF), human preferences over model outputs are used to train a rewar…