Knowledge Acquisition During Pre-training? Large Language Models Learn Better With Auxiliary Views

Joseph Lee, Yidi Huang, Dokyoon Kim +2

Gaps remain in our understanding of how large language models (LLMs) acquire knowledge during pre-training. We find that auxiliary views, conceptual reformulations, are key to this…