The Missing Minimal Pair: Stereotype Evaluation in LLMs

Nataliya Stepanova, Ivan Titov, Emily Allaway +1

A common approach to measuring bias in Large Language Models is to compare the log-likelihoods of two contrastive stereotype sentences. We argue that such single-pair comparisons a…