Thinking Augmented Pre-training

Liang Wang, Nan Yang, Shaohan Huang +2

This paper introduces a simple and scalable approach to improve the data efficiency of large language model (LLM) training by augmenting existing text data with thinking trajectori…