Benchmark Prediction with Sparse and Unpaired Cached Responses

Hayden Helm, Avijit Ghosh, Ben Johnson +1

Evaluating a generative model on all queries from all available benchmark tasks is prohibitively expensive. As such, models are often evaluated on different subsets of tasks and/or…