Best Arm Identification with LLM Judges and Limited Human

Ruicheng Ao, Hongyu Chen, Siyang Gao +2

We study fixed-confidence best-arm identification (BAI) where a cheap but potentially biased proxy (e.g., LLM judge) is available for every sample, while an expensive ground-truth…