9 papers
Ebisu: Benchmarking Large Language Models in Japanese Finance
Xueqing Peng, Ruoyu Xiang, Fan Zhang +9
Japanese finance combines agglutinative, head-final linguistic structure, mixed writing systems, and high-context communication norms that rely on indirect expression and implicit…
MedViz: An Agent-based, Visual-guided Research Assistant for Navigating Biomedical Literature
Huan He, Xueqing Peng, Yutong Xie +6
Biomedical researchers face increasing challenges in navigating millions of publications in diverse domains. Traditional search engines typically return articles as ranked text lis…
When Agents Trade: Live Multi-Market Trading Benchmark for LLM Agents
Lingfei Qian, Xueqing Peng, Yan Wang +14
Although Large Language Model (LLM)-based agents are increasingly used in financial trading, it remains unclear whether they can reason and adapt in live markets, as most studies t…
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application
Xueqing Peng, Lingfei Qian, Yan Wang +44
Real-world financial analysis involves information across multiple languages and modalities, from reports and news to scanned filings and meeting recordings. Yet most existing eval…
OrdRankBen: A Novel Ranking Benchmark for Ordinal Relevance in NLP
Yan Wang, Lingfei Qian, Xueqing Peng +2
The evaluation of ranking tasks remains a significant challenge in natural language processing (NLP), particularly due to the lack of direct labels for results in real-world scenar…
Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance
Xueqing Peng, Triantafillos Papadopoulos, Efstathia Soufleri +7
Despite Greece's pivotal role in the global economy, large language models (LLMs) remain underexplored for Greek financial context due to the linguistic complexity of Greek and the…