8 papers
SciPaths: Forecasting Pathways to Scientific Discovery
Eric Chamoun, Yizhou Chi, Yulong Chen +4
Scientific progress depends on sequences of enabling contributions, yet existing AI4Science benchmarks largely focus on citation prediction, literature retrieval, or idea generatio…
The Automatic Verification of Image-Text Claims (AVerImaTeC) Shared Task
Rui Cao, Zhenyun Deng, Yulong Chen +2
The Automatic Verification of Image-Text Claims (AVerImaTeC) shared task aims to advance system development for retrieving evidence and verifying real-world image-text claims. Part…
Multimodal Claim Extraction for Fact-Checking
Joycelyn Teo, Rui Cao, Zhenyun Deng +3
Automated Fact-Checking (AFC) relies on claim extraction as a first step, yet existing methods largely overlook the multimodal nature of today's misinformation. Social media posts…
FinMMDocR: Benchmarking Financial Multimodal Reasoning with Scenario Awareness, Document Understanding, and Multi-Step Computation
Zichen Tang, Haihong E, Rongjin Li +18
We introduce FinMMDocR, a novel bilingual multimodal benchmark for evaluating multimodal large language models (MLLMs) on real-world financial numerical reasoning. Compared to exis…
Beyond Accuracy: A Geometric Stability Analysis of Large Language Models in Chess Evaluation
Xidan Song, Weiqi Wang, Ruifeng Cao +1
The evaluation of Large Language Models (LLMs) in complex reasoning domains typically relies on performance alignment with ground-truth oracles. In the domain of chess, this standa…
AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Web
Rui Cao, Zifeng Ding, Zhijiang Guo +2
Textual claims are often accompanied by images to enhance their credibility and spread on social media, but this also raises concerns about the spread of misinformation. Existing d…