7 papers
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
Zhi Chen, Zhensu Sun, Yuling Shi +2
Repository-level performance-optimization benchmarks such as GSO, SWE-Perf and SWE-fficiency evaluate coding agents by applying patches to real repositories and comparing runtime a…
Same Signal, Different Semantics: A Cross-Framework Behavioral Analysis of Software Engineering Agents
Wei Ma, Zhi Chen, Jingxu Gu +3
Behavioral studies of LLM-based software engineering agents extract operational rules about which trajectory shapes correlate with higher resolution rates: that a test step follows…
Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents
Zhi Chen, Zhensu Sun, Yuling Shi +4
Large Language Model (LLM) code agents increasingly resolve repository-level issues by iteratively editing code, invoking tools, and validating candidate patches. In these workflow…
Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios
Zhi Chen, Wei Ma, Lingxiao Jiang
AI-driven software development has rapidly advanced with the emergence of software development agents that leverage large language models (LLMs) to tackle complex, repository-level…
Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios
Zhi Chen, Lingxiao Jiang
In recent years, AI-based software engineering has progressed from pre-trained models to advanced agentic workflows, with Software Development Agents representing the next major le…
Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization
Zhi Chen, Lingxiao Jiang
In the rapidly evolving field of machine learning, training models with datasets from various locations and organizations presents significant challenges due to privacy and legal c…
Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMs
Wei Ma, Zhihao Lin, Shangqing Liu +8
Code analysis is fundamental in Software Engineering, supporting debugging, optimization, and security assessment. Human developers approach it through syntax parsing, static seman…