9 papers
DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data
Zhen Huang, Yikun Wang, Shijie Xia +1
Pretraining data processing is critical to the downstream performance of Large Language Models (LLMs). However, many existing approaches define a fixed processing strategy at the c…
Diagnosing and Mitigating Context Rot in Long-horizon Search
Shijie Xia, Yikun Wang, Zhen Huang +1
Extensive context has become the norm as Large Language Models (LLMs) are increasingly deployed in long-horizon search tasks. The concern that increasing context length degrades mo…
Data Darwinism Part II: DataEvolve -- AI can Autonomously Evolve Pretraining Data Curation
Tiantian Mi, Dongming Shan, Zhen Huang +6
Data Darwinism (Part I) established a ten-level hierarchy for data processing, showing that stronger processing can unlock greater data value. However, that work relied on manually…
LIMO: Less is More for Reasoning
Yixin Ye, Zhen Huang, Yang Xiao +3
We challenge the prevailing assumption that complex reasoning in large language models (LLMs) necessitates massive training data. We demonstrate that sophisticated mathematical rea…
Interaction as Intelligence: Deep Research With Human-AI Partnership
Lyumanshan Ye, Xiaojie Cai, Xinkai Wang +23
This paper introduces "Interaction as Intelligence" research series, presenting a reconceptualization of human-AI relationships in deep research tasks. Traditional approaches treat…
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI
Zhen Huang, Zengzhi Wang, Shijie Xia +25
The evolution of Artificial Intelligence (AI) has been significantly accelerated by advancements in Large Language Models (LLMs) and Large Multimodal Models (LMMs), gradually showc…