Pair-Dependent Judge Errors Change the Minimax Rate of Semantic Entropy

Jiamiao Liu, Dewen Qiao, Yu Zhang +1

Semantic entropy, the entropy of the meanings among a language model's sampled answers, is estimated from the verdicts of a judge that declares pairs of answers equivalent or not,…