OpenCSG Chinese Corpus: A Series of High-quality Chinese Datasets for LLM Training

Yijiong Yu, Ziyun Dai, Zekun Wang +3

Large language models (LLMs) have demonstrated remarkable capabilities, but their success heavily relies on the quality of pretraining corpora. For Chinese LLMs, the scarcity of hi…