From the 1 of 30 linked papers with an AI index.
30 papers
MSQA: A Natively Sourced Multilingual and Multicultural SimpleQA Benchmark
Xianru Chen, Yukai Huang, Mingxiang Chen +6
The paper introduces MSQA, a benchmark of over a thousand native questions in 11 languages designed to evaluate whether multilingual models also understand the cultures behind thos…
Knowledge Index of Noah's Ark
Sheng Jin, Minghao Liu, Yunze Xiao +24
Knowledge benchmarks for LLMs face three issues: scaling-driven designs that do not operationalize disciplinary representativeness; flat-payment annotation that permits lazy consen…
EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies
Xavier Hu, Jinxiang Xia, Shengze Xu +13
Long-horizon planning is widely recognized as a core capability of autonomous LLM-based agents; however, current evaluation frameworks suffer from being largely episodic, domain-sp…
Understanding by Reconstruction: Reversing the Software Development Process for LLM Pretraining
Zhiyuan Zeng, Yichi Zhang, Yong Shan +11
While Large Language Models (LLMs) have achieved remarkable success in code generation, they often struggle with the deep, long-horizon reasoning required for complex software engi…
CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
Siyi Li, Jiajun Shi, Shiwen Ni +9
Large Reasoning Models (LRMs) have demonstrated strong performance by producing extended Chain-of-Thought (CoT) traces before answering. However, this paradigm often induces over-r…
Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization
Qianben Chen, Tianrui Qin, King Zhu +21
Recent deep research agents primarily improve performance by scaling reasoning depth, but this leads to high inference cost and latency in search-intensive scenarios. Moreover, gen…