Mapping Social Choice Theory to RLHF

Jessica Dai, Eve Fleisig

Recent work on the limitations of using reinforcement learning from human feedback (RLHF) to incorporate human preferences into model behavior often raises social choice theory as…