4 papers · 1 filter
LabOPBench: A Multimodal Benchmark for Evidence-Grounded Laboratory Decision Making
Jingyuan Li, Yujie Liu, Yijie Dai +18
Laboratory experiments involve more than following predefined protocols; they require adaptive decision-making as the experimental state changes. Existing scientific benchmarks rar…
Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
Wanghan Xu, Yuhao Zhou, Yifan Zhou +104
Despite advances in scientific AI, a coherent framework for Scientific General Intelligence (SGI)-the ability to autonomously conceive, investigate, and reason across scientific do…
DeepResearch Arena: The First Exam of LLMs' Research Abilities via Seminar-Grounded Tasks
Haiyuan Wan, Chen Yang, Junchi Yu +10
Deep research agents have attracted growing attention for their potential to orchestrate multi-stage research workflows, spanning literature synthesis, methodological design, and e…
Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning
Yuhao Zhou, Yiheng Wang, Xuming He +26
Scientific discoveries increasingly rely on complex multimodal reasoning based on information-intensive scientific data and domain-specific expertise. Empowered by expert-level sci…