Efficient MAP Estimation of LLM Judgment Performance with Prior Transfer

Huaizhi Qu, Inyoung Choi, Zhen Tan +6

LLM ensembles are widely used for LLM judges. However, how to estimate their accuracy, especially in an efficient way, is unknown. In this paper, we present a principled maximum a…