From the 1 of 6 linked papers with an AI index.
6 papers
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
Zhi Chen, Zhensu Sun, Yuling Shi +2
The paper audits three repository-level performance‑optimization benchmarks (GSO, SWE‑Perf, SWE‑efficiency) to assess how reliably they measure coding agents, revealing issues with…
Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMs
Wei Ma, Zhihao Lin, Shangqing Liu +8
Code analysis is fundamental in Software Engineering, supporting debugging, optimization, and security assessment. Human developers approach it through syntax parsing, static seman…
Same Signal, Different Semantics: A Cross-Framework Behavioral Analysis of Software Engineering Agents
Wei Ma, Zhi Chen, Jingxu Gu +3
Behavioral studies of LLM-based software engineering agents extract operational rules about which trajectory shapes correlate with higher resolution rates: that a test step follows…
Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents
Zhi Chen, Zhensu Sun, Yuling Shi +4
Large Language Model (LLM) code agents increasingly resolve repository-level issues by iteratively editing code, invoking tools, and validating candidate patches. In these workflow…
Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios
Zhi Chen, Wei Ma, Lingxiao Jiang
AI-driven software development has rapidly advanced with the emergence of software development agents that leverage large language models (LLMs) to tackle complex, repository-level…
Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios
Zhi Chen, Lingxiao Jiang
In recent years, AI-based software engineering has progressed from pre-trained models to advanced agentic workflows, with Software Development Agents representing the next major le…