Understanding Data Temporality Impact on Large Language Models Pre-training

Hippolyte Pilchen, Romain Fabre, Franck Signe Talla +2

Large language models (LLMs) are typically trained on shuffled corpora, yielding models whose knowledge is frozen at train time and whose temporal grounding remains poorly understo…