Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training

Jiawen Tao, Miao Peng, Yaoming Li +7

Synthetic textbook data has improved language model pre-training, but prior work largely treats the benefit as a property of generated content or local rewriting style. We study a…