UltraData-Code: From Raw Repositories to Algorithmically Dense and Task-Oriented Code Data

Chengying Tu, Hengyu Zhao, Shuaikang Xue +13

Code generation has become a core capability of large language models (LLMs), and code data is central to the pre-training process that develops it. As obtaining public code of suf…