7 papers · 1 filter
MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning
Run-Ze Fan, Zengzhi Wang, Pengfei Liu
Scientific reasoning is critical for developing AI scientists and supporting human researchers in advancing the frontiers of natural science discovery. However, the open-source com…
OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
Zengzhi Wang, Fan Zhou, Xuefeng Li +1
Different base language model families, such as Llama and Qwen, exhibit divergent behaviors during post-training with reinforcement learning (RL), especially on reasoning-intensive…
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI
Zhen Huang, Zengzhi Wang, Shijie Xia +25
The evolution of Artificial Intelligence (AI) has been significantly accelerated by advancements in Large Language Models (LLMs) and Large Multimodal Models (LMMs), gradually showc…
MathPile: A Billion-Token-Scale Pretraining Corpus for Math
Zengzhi Wang, Xuefeng Li, Rui Xia +1
High-quality, large-scale corpora are the cornerstone of building foundation models. In this work, we introduce MathPile, a diverse and high-quality math-centric corpus comprising…
Data Contamination Report from the 2024 CONDA Shared Task
Oscar Sainz, Iker GarcÃa-Ferrero, Alon Jacovi +25
The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as…
OlympicArena Medal Ranks: Who Is the Most Intelligent AI So Far?
Zhen Huang, Zengzhi Wang, Shijie Xia +1
In this report, we pose the following question: Who is the most intelligent AI model to date, as measured by the OlympicArena (an Olympic-level, multi-discipline, multi-modal bench…