12 papers
EvolvAttack: Backdooring self Evolving VLM Agents
Chuan Zou, Yang Xiang, Zexin Li +2
Self-evolving mechanisms let vision-language model (VLM) agents continually adapt through evolving memory, but they also allow malicious behavior to emerge only after deployment. W…
Adversarial Testing of Automated Program Repair Agents for Security Vulnerabilities
Fares Trad, Simin Chen, Hung Viet Pham +2
Software agents with Large Language Models (LLMs) are designed for Automated Program Repair (APR) tasks, raising the possibility that, in the near future, APR agents will fix bugs…
When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs
Tong Zhang, Zexin Li, Simin Chen +1
Jailbreak defenses are essential for protecting large language models (LLMs), but they can also introduce secondary costs that weaken model utility. We present a systematic study o…
Trustworthy AI Software Engineers
Aldeida Aleti, Baishakhi Ray, Rashina Hoda +1
With the rapid rise of AI coding agents, the fundamental premise of what it means to be a software engineer is in question. In this vision paper, we examine what it means for an AI…
SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning
Jinjun Peng, Magnus Saebo, Tianjun Zhong +6
The deployment of coding agents in privacy-sensitive and resource-constrained environments drives the demand for capable open-weight Small Language Models (SLMs). However, they suf…
Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers
Simin Chen, Jinjun Peng, Yixin He +2
Deep learning (DL) compilers are core infrastructure in modern DL systems, offering flexibility and scalability beyond vendor-specific libraries. This work uncovers a fundamental v…
AppForge: From Assistant to Independent Developer -- Are GPTs Ready for Software Development?
Dezhi Ran, Yuan Cao, Mengzhou Wu +10
Large language models (LLMs) have demonstrated remarkable capability in function-level code generation tasks. Unlike isolated functions, real-world applications demand reasoning ov…
REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement
Anvith Pabba, Simin Chen, Alex Mathai +2
Large Language Models (LLMs) have recently shown strong potential in automatic program repair (APR), especially in repository-level settings where the goal is to generate patches b…
Red Teaming Program Repair Agents: When Correct Patches can Hide Vulnerabilities
Simin Chen, Yixin He, Suman Jana +1
LLM-based agents are increasingly deployed for software maintenance tasks such as automated program repair (APR). APR agents automatically fetch GitHub issues and use backend LLMs…
CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning
Monoshi Kumar Roy, Simin Chen, Benjamin Steenhoek +4
Understanding and reasoning about code semantics is essential for enhancing code LLMs' abilities to solve real-world software engineering (SE) tasks. Although several code reasonin…
Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination
Simin Chen, Pranav Pusarla, Baishakhi Ray
The rapid evolution of code largelanguage models underscores the need for effective and transparent benchmarking of their reasoning capabilities. However, the current benchmarking…
Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation
Simin Chen, Yiming Chen, Zexin Li +8
Data contamination has received increasing attention in the era of large language models (LLMs) due to their reliance on vast Internet-derived training corpora. To mitigate the ris…