1 citations · 1 across the 10 of their papers we have counts for
11 papers
Zero2Repo: Can Coding Agents Build Repositories from Scratch?
Pei Yang, Tianyu Shi, Yuhang Yao +23
Coding agents are increasingly asked to build software rather than patch it, yet benchmarks for from-scratch repository construction are mostly limited to a single language and dep…
Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
Lin Shi, Haowei Lin, Zixuan Zhu +123
Evaluating agents on the growing number of agentic benchmarks is challenging because they often require complex environments and agent integrations. We introduce Harbor Adapters, a…
SWE-chat: Coding Agent Interactions From Real Users in the Wild
Joachim Baumann, Vishakh Padmakumar, Xiang Li +3
AI coding agents are being adopted at scale, yet we lack empirical evidence on how people actually use them and how much of their output is useful in practice. We present SWE-chat,…
Trojan's Whisper: Stealthy Manipulation of OpenClaw through Injected Bootstrapped Guidance
Fazhong Liu, Zhuoyan Chen, Tu Lan +6
Autonomous coding agents are increasingly integrated into software development workflows, offering capabilities that extend beyond code suggestion to active system interaction and…
SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding
Songcheng Cai, Zhiheng Lyu, Yuansheng Ni +13
Agentic repository-level code understanding is essential for automating complex software engineering tasks, yet the field lacks reliable benchmarks. Existing evaluations often over…
AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis
Pei Yang, Wanyi Chen, Asuka Yuxi Zheng +13
Large language model (LLM) agents offer a promising data-driven approach to automating Site Reliability Engineering (SRE), yet their enterprise deployment is constrained by three c…
HerAgent: Rethinking the Automated Environment Deployment via Hierarchical Test Pyramid
Xiang Li, Siyu Lu, Federica Sarro +2
Automated software environment setup is a prerequisite for testing, debugging, and reproducing failures, yet remains challenging in practice due to complex dependencies, heterogene…
Environment-in-the-Loop: Rethinking Code Migration with LLM-based Agents
Xiang Li, Zhiwei Fei, Ying Ma +3
Modern software systems continuously undergo code upgrades to enhance functionality, security, and performance, and Large Language Models (LLMs) have demonstrated remarkable capabi…
JMigBench: A Benchmark for Evaluating LLMs on Source Code Migration (Java 8 to Java 11)
Nishil Amin, Zhiwei Fei, Xiang Li +2
We build a benchmark to evaluate large language models (LLMs) for source code migration tasks, specifically upgrading functions from Java 8 to Java 11. We first collected a dataset…
Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories
Oorja Majgaonkar, Zhiwei Fei, Xiang Li +2
The increasing deployment of Large Language Model (LLM) agents for complex software engineering tasks has created a need to understand their problem-solving behaviours beyond simpl…
Prometheus: Towards Long-Horizon Codebase Navigation for Repository-Level Problem Solving
Yue Pan, Zimin Chen, Siyu Lu +8
Large Language Models (LLMs) have shown remarkable capabilities in automating software engineering tasks, spurring the emergence of coding agents that scaffold LLMs with external t…