Speculative Evaluation of Stochastic LLMs

Qianli Shen, Xiang Li, Ruomeng Ding +3

Evaluating a stochastic large language model is costly: benchmark scores estimate expected performance from randomized rollouts, yet uniform repetition ignores sharp differences in…