Mid-Training of Large Language Models: A Survey

Kaixiang Mo, Yuxin Shi, Weiwei Weng +4

Large language models (LLMs) are typically developed through large-scale pre-training followed by task-specific fine-tuning. Recent advances highlight the importance of an intermed…