TRUCE: Private Benchmarking to Prevent Contamination and Improve Comparative Evaluation of LLMs

Tanmay Rajore, Nishanth Chandran, Sunayana Sitaram +4

Benchmarking is the de-facto standard for evaluating LLMs, due to its speed, replicability and low cost. However, recent work has pointed out that the majority of the open source b…