3 papers
cs.CL2025
Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models
Jijie Li, Li Du, Hanyu Zhao +5
Large Language Models (LLMs) demonstrate strong performance in real-world applications, yet existing open-source instruction datasets often concentrate on narrow domains, such as m…
cs.CL2025
CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large Language Models
Guang Liu, Liangdong Wang, Jijie Li +6
We introduce CCI4.0, a large-scale bilingual pre-training dataset engineered for superior data quality and diverse human-like reasoning trajectory. CCI4.0 occupies roughly TB…
cs.LG2025
InCo-DPO: Balancing Distribution Shift and Data Quality for Enhanced Preference Optimization
Yunan Wang, Jijie Li, Bo-Wen Zhang +2
Direct Preference Optimization (DPO) optimizes language models to align with human preferences. Utilizing on-policy samples, generated directly by the policy model, typically resul…