5 papers
Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning
Junpeng Ding, Zichen Tang, Haihong E +17
We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from…
AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images
Bo Zhang, Tzu-Yen Ma, Zichen Tang +18
We introduce AEGIS, A holistic benchmark for Evaluating forensic analysis of AI-Generated academic ImageS. Compared to existing benchmarks, AEGIS features three key advances: (1) D…
RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems
Jiacheng Liu, Zichen Tang, Zhongjun Yang +8
People commonly leverage structured content to accelerate knowledge acquisition and research problem solving. Among these, roadmaps guide researchers through hierarchical subtasks…
Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning
Rongjin Li, Zichen Tang, Xianghe Wang +9
With the rapid progress of multimodal large language models (MLLMs), AI already performs well at literature retrieval and certain reasoning tasks, serving as a capable assistant to…
THEMIS: Towards Holistic Evaluation of MLLMs for Scientific Paper Fraud Forensics
Tzu-Yen Ma, Bo Zhang, Zichen Tang +20
We present THEMIS, a novel multi-task benchmark designed to comprehensively evaluate multimodal large language models (MLLMs) on visual fraud reasoning within real-world academic s…