MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models

Jiazheng Li, Lu Yu, Qing Cui +4

High-quality data plays a critical role in the pretraining and fine-tuning of large language models (LLMs), even determining their performance ceiling to some degree. Consequently,…