6 papers · 1 filter
Trustworthy AI Software Engineers
Aldeida Aleti, Baishakhi Ray, Rashina Hoda +1
With the rapid rise of AI coding agents, the fundamental premise of what it means to be a software engineer is in question. In this vision paper, we examine what it means for an AI…
CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning
Monoshi Kumar Roy, Simin Chen, Benjamin Steenhoek +4
Understanding and reasoning about code semantics is essential for enhancing code LLMs' abilities to solve real-world software engineering (SE) tasks. Although several code reasonin…
AppForge: From Assistant to Independent Developer -- Are GPTs Ready for Software Development?
Dezhi Ran, Yuan Cao, Mengzhou Wu +10
Large language models (LLMs) have demonstrated remarkable capability in function-level code generation tasks. Unlike isolated functions, real-world applications demand reasoning ov…
REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement
Anvith Pabba, Simin Chen, Alex Mathai +2
Large Language Models (LLMs) have recently shown strong potential in automatic program repair (APR), especially in repository-level settings where the goal is to generate patches b…
Red Teaming Program Repair Agents: When Correct Patches can Hide Vulnerabilities
Simin Chen, Yixin He, Suman Jana +1
LLM-based agents are increasingly deployed for software maintenance tasks such as automated program repair (APR). APR agents automatically fetch GitHub issues and use backend LLMs…
Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination
Simin Chen, Pranav Pusarla, Baishakhi Ray
The rapid evolution of code largelanguage models underscores the need for effective and transparent benchmarking of their reasoning capabilities. However, the current benchmarking…