Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts

Jacob Haimes, Cenny Wenner, Kunvar Thaman +4

The training data for many Large Language Models (LLMs) is contaminated with test data. This means that public benchmarks used to assess LLMs are compromised, suggesting a performa…