4 papers
FINCOSMOS: An Open-World Evaluation Paradigm for Agents in Finance
Yupeng Cao, Xueqing Peng, Lingfei Qian +14
Language agents increasingly work for users on the open Web, and finance is where trusting them is hardest. The data must come from the Web, the strategy lives in the user's head,…
LLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting
Lingfei Qian, Yan Wang, Xueqing Peng +12
Financial backtests of large language models (LLMs) are meaningful only if the model did not already know the outcomes it is asked to predict. Practitioners guard against this by t…
Herculean: An Agentic Benchmark for Financial Intelligence
Xueqing Peng, Zhuohan Xie, Yupeng Cao +60
As AI agents advance, the central question is no longer whether they can solve isolated financial tasks, but whether their capabilities generalize across complete professional work…
FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Tasks
Yupeng Cao, Haohang Li, Weijin Liu +11
Recent studies demonstrate that tool-calling capability enables large language models (LLMs) to interact with external environments for long-horizon financial tasks. While existing…