Contamination Inflates Scores but Rarely Reorders Large Language Model Leaderboards

Xingyao Xiao, Yihong Cheng

Benchmark contamination, the leakage of test items into training data, is widely described as a threat to the reliability of large language model (LLM) leaderboards. We argue that…