Training on the Benchmark Is Not All You Need

Shiwen Ni, Xiangtao Kong, Chengming Li +4

The success of Large Language Models (LLMs) relies heavily on the huge amount of pre-training data learned in the pre-training phase. The opacity of the pre-training process and th…