4 papers
Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models
Xiaoyu Luo, Tao Ren, Wenrui Yu +3
The rapid capability gains of frontier language models are widely attributed to improved reasoning abilities, yet this cannot be verified as raw CoT traces in closed-source systems…
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +378
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models
Yiwei Liu, Yucheng Li, Xiao Li +1
Joint logical-numerical reasoning remains a major challenge for language models, yet existing datasets rely on fixed rule sets and offer limited control over task complexity, const…
FormulaReasoning: A Dataset for Formula-Based Numerical Reasoning
Xiao Li, Bolin Zhu, Kaiwen Shi +4
The application of physics formulas is a fundamental human capability in numerical reasoning. While existing datasets often rely on implicit mathematical knowledge, they rarely exp…