Efficient Cost-Aware LLM Evaluation via Bayesian Bandit Gittins Indices

Qian Xie, Yueli He, Nairen Cao

Exhaustively evaluating every candidate LLM configuration on every benchmark item to identify a high-performing one is costly. We formulate configuration selection as a cost-aware…