Using tournaments to calculate AUROC for zero-shot classification with LLMs

WonJin Yoon, Ian Bulovic, Timothy A. Miller

Large language models perform surprisingly well on many zero-shot classification tasks, but are difficult to fairly compare to supervised classifiers due to the lack of a modifiabl…