10 papers
Unlocking Model Potentials Through Adaptive Multi-Agent Scaffolding for Efficient Issue Resolution
Yang Chen, Aliya Ahmad, Yiheng Zhou +1
Resolving issues with ambiguous and incomplete descriptions, particularly concerning complex bugs, requires a sophisticated, long-horizon workflow. Agents must navigate codebases t…
Can Old Tests Do New Tricks for Resolving SWE Issues?
Yang Chen, Toufique Ahmed, Reyhaneh Jabbarvand +1
Test suites in real-world projects are often large and achieve high code coverage, yet they remain insufficient for detecting all bugs. The abundance of unresolved issues in open-s…
Evaluating Code Reasoning Abilities of Large Language Models Under Real-World Settings
Changshu Liu, Alireza Ghazanfari, Yang Chen +1
Code reasoning tasks are becoming prevalent in large language model (LLM) assessments. Yet, there is a dearth of studies on the impact of real-world complexities on code reasoning,…
Process-Centric Analysis of Agentic Software Systems
Shuyang Liu, Yang Chen, Rahul Krishna +3
Agentic systems are modern software systems: they consist of orchestrated modules, expose interfaces, and are deployed in software pipelines. Unlike conventional programs, their ex…
Assessing Coherency and Consistency of Code Execution Reasoning by Large Language Models
Changshu Liu, Yang Chen, Reyhaneh Jabbarvand
This paper proposes CES, a task to evaluate the abilities of LLMs in simulating program execution and using that reasoning in programming tasks. Besides measuring the correctness o…
SysTradeBench: An Iterative Build-Test-Patch Benchmark for Strategy-to-Code Trading Systems with Drift-Aware Diagnostics
Yuchen Cao, Hanlin Zhang, Jacky Wai Keung +2
Large language models (LLMs) are increasingly used as quantitative research copilots to translate natural-language strategy specifications into executable trading code. Yet most ex…