AI Safety via Debate is Compromised by Cognitive Biases

Gefei Liu, Sonya Rashkovan, Sophia Lloyd George +2

Reinforcement learning from human feedback (RLHF) has played a central role in making large language models responsive to human instructions. However, human evaluators often favor…