5 papers
DRAGON: Domain-specific Robust Automatic Data Generation for RAG Optimization
Haiyang Shen, Hang Yan, Zhongshi Xing +6
Retrieval-augmented generation (RAG) can substantially enhance the performance of LLMs on knowledge-intensive tasks. Various RAG paradigms - including vanilla, planning-based, and…
Agent Bain vs. Agent McKinsey: A New Text-to-SQL Benchmark for the Business Domain
Yue Li, Ran Tao, Derek Hommel +4
Text-to-SQL benchmarks have traditionally only tested simple data access as a translation task of natural language to SQL queries. But in reality, users tend to ask diverse questio…
FaMA: LLM-Empowered Agentic Assistant for Consumer-to-Consumer Marketplace
Yineng Yan, Xidong Wang, Jin Seng Cheng +5
The emergence of agentic AI, powered by Large Language Models (LLMs), marks a paradigm shift from reactive generative systems to proactive, goal-oriented autonomous agents capable…
OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?
Xuetian Chen, Yinghao Chen, Xinfeng Yuan +12
Computer-using agents have shown strong potential to boost human productivity and enable new application forms across platforms. While recent advances have led to usable applicatio…
ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents
Haiyang Shen, Yue Li, Desong Meng +5
Recent advancements in integrating large language models (LLMs) with application programming interfaces (APIs) have gained significant interest in both academia and industry. Recen…