8 papers
Right Frame, Wrong Rule: Cultural Cues Expose the Financial Knowledge Gap They Were Meant to Close
Rania Elbadry, Ahmed Heakl, Saeed Almheiri +12
When a question has valid answers under different normative frameworks, a language model must decide which framework to use and whether it can answer correctly within it. We call t…
FinExam-10K: When Retrieval Helps Financial Reasoning?
Yan Lin, Jingyu Sun, Zhongliang Guo +3
Professional financial examinations require models to combine domain knowledge, calculation, and judgment, yet no benchmark covers the full CFA and FRM structure under one protocol…
Seeing Is Not Deciding: Can Multimodal LLMs Act as Effective CEOs?
Yuyang Dai, Xueqing Peng, Yuxia Wang +2
Large language models are increasingly applied as autonomous decision-making agents. However, in executive business decisions, existing benchmarks are limited to textonly settings.…
Overview of FinMMEval 2026 Task 2: Multilingual Financial Short-Answer Question Answering
Zhuohan Xie, Xueqing Peng, Georgi Georgiev +18
FinMMEval 2026 Task 2 evaluates short-answer financial question answering over multilingual evidence. Each final-test item pairs an English question with financial statements and n…
Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering
Zhuohan Xie, Yuyang Dai, Rania Elbadry +18
FinMMEval 2026 Task 1 evaluates multilingual financial multiple-choice question answering in English, Chinese, Arabic, and Hindi. The task tests whether systems can select the corr…
AI YOU Town: Make Friends and Money with Your Digital Twin
Yan Lin, Yuyang Dai, Jiahui Geng +1
Existing approaches to infer user traits and generate responses consistent with a persona rely on static prompting. They lack calibrated uncertainty, ignore sequential evidence, an…
The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems
Zhuohan Xie, Rania Elbadry, Fan Zhang +12
We present the setup and the tasks of the FinMMEval Lab at CLEF 2026, which introduces the first multilingual and multimodal evaluation framework for financial Large Language Model…
RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid?
Yuyang Dai, Yan Lin, Zhuohan Xie +1
Reliable financial reasoning requires knowing not only how to answer, but also when an answer cannot be justified. In real financial practice, problems often rely on implicit assum…