1 citations · 1 across the 8 of their papers we have counts for
6 papers · 1 filter
UltraData-Code: From Raw Repositories to Algorithmically Dense and Task-Oriented Code Data
Chengying Tu, Hengyu Zhao, Shuaikang Xue +13
Code generation has become a core capability of large language models (LLMs), and code data is central to the pre-training process that develops it. As obtaining public code of suf…
UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing
Xinlong Zhao, Dongsheng Liu, Hengyu Zhao +9
As available training data approaches its physical limit, gains from Scaling Laws have begun to diminish. Consequently, improving Large Language Models (LLMs) now depends less on d…
MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
MiniCPM Team, Wenhao An, Yingfa Chen +44
The evolution of large language models (LLMs) towards applications with ultra-long contexts faces challenges posed by the high computational and memory costs of the Transformer arc…
Data Science and Technology Towards AGI Part I: Tiered Data Management
Yudong Wang, Zixuan Fu, Hengyu Zhao +14
The development of artificial intelligence can be viewed as an evolution of data-driven learning paradigms, with successive shifts in data organization and utilization continuously…
FactNet: A Billion-Scale Knowledge Graph for Multilingual Factual Grounding
Yingli Shen, Wen Lai, Jie Zhou +7
Large language models hallucinate factual claims and struggle to ground their outputs in retrievable evidence, particularly in non-English languages. Existing resources impose a tr…
JudgeRLVR: Judge First, Generate Second for Efficient Reasoning
Jiangshan Duo, Hanyu Li, Hailin Zhang +3
Reinforcement Learning with Verifiable Rewards (RLVR) has become a standard paradigm for reasoning in Large Language Models. However, optimizing solely for final-answer correctness…