15 papers
FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?
Jingpu Yang, Fengxian Ji, Jinri Guo +7
Financial scenarios are diverse and complex, spanning varying data conditions, tool configurations, and workflows. Yet existing CUA, Computer-Using Agent, evaluation tasks remain l…
Can Agentic Trading Systems Pay for Their Own Intelligence?
Qiqi Duan, Changlun Li, Chen Wang +10
Large language model (LLM) agents are increasingly used in trading systems, where model reasoning, tool use, and continual decisions incur costs that are expected to produce tradin…
Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation
Fengxian Ji, Yuke Li, Jingpu Yang +8
However, whether these judges truly evaluate the scientific substance of ideas or are influenced by superficial stylistic presentation remains an open question. To address this que…
Financial Evidence Crowding: Diagnosing and Mitigating Constraint-Induced Displacement in Retrieval-Augmented Generation
Yixi Zhou, Jiayi Yin, Fan Zhang +2
Retrieval-augmented generation (RAG) retrieves candidate evidence and sends only a limited top-ranked subset, the top-k context, to a generator. In financial question answering, pa…
LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents
Jingpu Yang, Fengxian Ji, Zhengzhao Lai +8
Scientific embodied agents are increasingly capable of carrying out laboratory procedures, but executing these procedures safely in dynamic laboratory environments remains challeng…
Parametric Memory Decoding for Zero-Shot Routing in LoRA-Based External Parametric Memory
Fengxian Ji, Zhuohan Xie, Jingpu Yang +3
With the rise of parametric memory, LoRA-based External Parametric Memory (EPM) has emerged as a modular solution, but existing routing methods often introduce additional training,…
FinPersona-Bench: A Benchmark for Longitudinal Psychometric Stability of Autonomous Financial Agents
Muhammad Usman Safder, Ayesha Gull, Rania Elbadry +7
Large Language Models (LLMs) are increasingly deployed as autonomous financial agents initialized with explicit behavioral mandates such as "preserve capital" or "avoid speculative…
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosures
Fan Zhang, Mingzi Song, Rania Elbadry +20
Financial reporting systems increasingly leverage Large Language Models (LLMs) to extract and summarize corporate disclosures. However, most existing approaches assume a single-mar…
When Alpha Disappears: A One-Switch Benchmark for Decision-Time Leakage in Financial Backtests
Fan Zhang, Zhen Li, Sijia Peng +1
We introduce When Alpha Disappears, a paired evaluation benchmark for diagnosing decision-time leakage in financial machine-learning backtests. Rather than treating leakage as a bi…
REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading
Chengshuai Zhao, Fan Zhang, Kumar Satvik Chaudhary +4
Open-ended grading is central to equitable and personalized education, yet manual grading remains time-consuming and costly, underscoring the need for automated grading systems. Al…
SQLStructEval: Structural Evaluation of LLM Text-to-SQL Generation
Yixi Zhou, Fan Zhang, Zhiqiao Guo +4
In Text-to-SQL tasks, large language models can generate structurally different SQL queries that return correct answers for the same intent. We call this phenomenon execution-corre…
Watermarking LLM Agent Trajectories
Wenlong Meng, Chen Gong, Terry Yue Zhuo +6
LLM agents rely heavily on high-quality trajectory data to guide their problem-solving behaviors, yet producing such data requires substantial task design, high-capacity model gene…
FinCARDS: Card-Based Analyst Reranking for Financial Document Question Answering
Yixi Zhou, Fan Zhang, Yu Chen +3
Financial question answering (QA) over long corporate filings requires evidence to satisfy strict constraints on entities, financial metrics, fiscal periods, and numeric values. Ho…
FinCriticalED: A Visual Benchmark for Financial Fact-Level OCR
Yueru He, Xueqing Peng, Yupeng Cao +13
Recent progress in multimodal large language models (MLLMs) has substantially improved document understanding, yet strong optical character recognition (OCR) performance on surface…
EngTrace: A Symbolic Benchmark for Verifiable Process Evaluation of Engineering Reasoning
Ayesha Gull, Muhammad Usman Safder, Rania Elbadry +4
Most engineering problems involve complex multi-step derivations through checkable intermediate quantities. Grading only the final answer misjudges LLMs, crediting correct answers…