10 papers
DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations
Jiazhen Jiang, Boxi Cao, Lingyong Yan +6
As autonomous agents rapidly evolve, their ability to reliably manipulate ubiquitous digital documents has become critical for enabling general-purpose AI assistants and automating…
Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery
Tianyun Zhong, Wangyi Jiang, Wei Wang +15
Large language models (LLMs) excel at answering pre-specified questions, yet their ability to navigate the open-ended, pre-conclusion stage of discovery remains largely unmeasured.…
LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation
Dong Xu, Jialun Cao, Guozhao Mo +9
Formal specification is essential for rigorous program verification, yet writing correct specifications remains costly and difficult to automate. Although large language models (LL…
All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG
Dan Wang, Guozhao Mo, Yafei Shi +9
Multilingual Retrieval-Augmented Generation (mRAG) leverages cross-lingual evidence to ground Large Language Models (LLMs) in global knowledge. However, we show that current mRAG s…
DeepPresenter: Environment-Grounded Reflection for Agentic Presentation Generation
Hao Zheng, Guozhao Mo, Xinru Yan +7
Presentation generation requires deep content research, coherent visual design, and iterative refinement based on observation. However, existing presentation agents often rely on p…
LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?
Guozhao Mo, Wenliang Zhong, Jiawei Chen +7
Model Context Protocol (MCP) has become a key infrastructure for connecting LLMs with external tools, scaling to 10,000+ MCP servers with diverse tools. Unfortunately, there is sti…