4 papers · 1 filter
Hint Tuning: Less Data Makes Better Reasoners
Siqi Fan, Minghao Li, Xiaoqian Ma +6
Large reasoning models achieve high accuracy through extended chain-of-thought but generate 5--8 more tokens than necessary, applying verbose reasoning uniformly regardless of prob…
BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing?
Guoxin Chen, Fanzhe Meng, Jiale Zhao +12
Current code-agent benchmarks primarily evaluate localized issue resolution within a single target repository, leaving under-tested many software engineering tasks that require ext…
ShoppingComp: Are LLMs Really Ready for Your Shopping Cart?
Huaixiao Tou, Ying Zeng, Yuemeng Li +6
We present ShoppingComp, a challenging real-world benchmark for comprehensively evaluating LLM-powered shopping agents on three core capabilities: precise product retrieval, expert…
ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks
Minghao Li, Ying Zeng, Zhihao Cheng +2
The advent of Deep Research agents has substantially reduced the time required for conducting extensive research tasks. However, these tasks inherently demand rigorous standards of…