Fisher Random Walk: Automatic Debiasing Contextual Preference Inference for Large Language Model Evaluation

Yichi Zhang, Alexander Belloni, Ethan X. Fang +2

Motivated by the need for rigorous and scalable evaluation of large language models, we study contextual preference inference for pairwise comparison functionals of context-depende…