Wrong and More Confident: A Field Experiment on Large Language Models Taking a Graduate Economics Exam

Piyush Akimitsu

A red herring, an irrelevant passage added to a problem, corrupts a language model's reasoning and, through it, its final answer, while the form of the response survives untouched.…