Oasis: Data Curation and Assessment System for Pretraining of Large Language Models

Tong Zhou, Yubo Chen, Pengfei Cao +3

Data is one of the most critical elements in building a large language model. However, existing systems either fail to customize a corpus curation pipeline or neglect to leverage c…