Surface Form Competition: Why the Highest Probability Answer Isn't Always Right

Ari Holtzman, Peter West, Vered Shwartz +2

Large language models have shown promising results in zero-shot settings (Brown et al.,2020; Radford et al., 2019). For example, they can perform multiple choice tasks simply by co…