9 papers · 1 filter
RealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents
Yunqi Liu, Peiyu Wang, Bingli Wang +21
Command-line interface (CLI) agents increasingly execute longer-horizon tasks, yet existing benchmarks either assume that user instructions remain fixed throughout execution or eva…
AutoDataBench: A Data-centric Testbed for Accelerating Auto Research
Ruifeng Yuan, Yizhi Li, Yaxin Du +9
Existing auto-research benchmarks often entangle multiple sources of improvement, including training frameworks, hyperparameters, compute budgets, and data, making it difficult to…
ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
Siwei Wu, Jincheng Ren, Yizhi Li +12
Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve execution mechanisms from experience.…
HyperTool: Beyond Step-Wise Tool Calls for Tool-Augmented Agents
Yaxin Du, Yifan Zhou, Yujie Ge +7
Tool-augmented LLM agents commonly rely on step-wise atomic tool calls, where each invocation, observation, and value transfer is exposed in the main reasoning trace. This creates…
Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environments
Siwei Wu, Yizhi Li, Yuyang Song +8
Training agentic models for terminal-based tasks critically depends on high-quality terminal trajectories that capture realistic long-horizon interactions across diverse domains. H…
M2G-Eval: Enhancing and Evaluating Multi-granularity Multilingual Code Generation
Fanglin Xu, Wei Zhang, Jian Yang +5
The rapid advancement of code large language models (LLMs) has sparked significant research interest in systematically evaluating their code generation capabilities, yet existing b…
Context as a Tool: Context Management for Long-Horizon SWE-Agents
Shukai Liu, Jian Yang, Bo Jiang +4
Agents based on large language models have recently shown strong potential on real-world software engineering (SWE) tasks that require long-horizon interaction with repository-scal…
Scaling Laws for Code: Every Programming Language Matters
Jian Yang, Shawn Guo, Lin Jing +8
Code large language models (Code LLMs) are powerful but costly to train, with scaling laws predicting performance from model size, data, and compute. However, different programming…
REAL-Prover: Retrieval Augmented Lean Prover for Mathematical Reasoning
Ziju Shen, Naohao Huang, Fanyi Yang +11
Nowadays, formal theorem provers have made monumental progress on high-school and competition-level mathematics, but few of them generalize to more advanced mathematics. In this pa…