Predictions from language models for multiple-choice tasks are not robust under variation of scoring methods

Polina Tsvilodub, Hening Wang, Sharon Grosch +1

This paper systematically compares different methods of deriving item-level predictions of language models for multiple-choice tasks. It compares scoring methods for answer options…