Training Language Models to Fail Gracefully

Daman Arora, Xuecheng Liu, Himanshu Gaurav Singh +6

Reinforcement learning makes language models better at solving tasks, but the same mechanism also rewards guessing. When success is the only way to earn reward, opting for a safe f…