From the 1 of 29 linked papers with an AI index.
12 papers · 1 filter
Right Answer, Wrong Method: Shortcut Hacking Misleads the Evaluation of LLM Reasoning on Frontier Science Benchmarks
Xuan Ren, Weiqi Zhai, Tianle Pu +3
Scientific reasoning benchmarks typically evaluate large language models (LLMs) using final-answer accuracy. However, a correct answer does not necessarily demonstrate the reasonin…
FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning
Zeyu Wang, Jingye Xu, Xiaogang Li +7
Current multimodal benchmarks for scientific reasoning primarily evaluate local information extraction -- models recognize symbols and values and then perform textual inference. Th…
BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs
Ben Wang, Xiaogang Li, Ruochen Gao +6
Current multimodal models handle static image recognition well, but intuitive physical reasoning remains a weakness. Predicting how objects will move and interact from a single ima…
Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem
Weixun Wang, XiaoXiao Xu, Wanhe An +86
Agentic crafting requires LLMs to operate in real-world environments over multiple turns by taking actions, observing outcomes, and iteratively refining artifacts. Despite its impo…
Benchmark Health Index: A Systematic Framework for Benchmarking the Benchmarks of LLMs
Longyuan Zhu, Hairan Hua, Linlin Miao +1
Large Language Models (LLMs) are advancing rapidly, yet the benchmarks used to measure this progress are becoming increasingly unreliable. Score inflation and selective reporting h…
Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning
Zhengbo Jiao, Shaobo Wang, Zifan Zhang +4
Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet how visual evidence is…