6 papers
PatchBench: Evaluating AI Agents for Vulnerability Patching
Chihao Shen, Jiacheng Li, Aastha Mahajan +3
AI agents have recently demonstrated strong performance in automated vulnerability patching. However, existing evaluations often validate a patch only by testing whether the provid…
Symbolon: Symbolic Execution by Learning Code Transformation
Jie Zhu, Penghui Li, Zhongxuan Li +4
Symbolic execution is a powerful program analysis technique with broad applications, such as vulnerability detection, security testing, and malware analysis. However, this techniqu…
Benchmarking Correctness and Security in Multi-Turn Code Generation
Ruchit Rawal, Jeffrey Yang Fan Chiang, Chihao Shen +4
AI coding assistants powered by large language models (LLMs) have transformed software development, significantly boosting productivity. While existing benchmarks evaluate the corr…
Locus: Agentic Predicate Synthesis for Directed Fuzzing
Jie Zhu, Chihao Shen, Ziyang Li +3
Directed fuzzing aims to find program inputs that lead to specified target program states. It has broad applications, such as debugging system crashes, confirming reported bugs, an…
Towards Evaluating Proactive Risk Awareness of Multimodal Language Models
Youliang Yuan, Wenxiang Jiao, Yuejin Xie +5
Human safety awareness gaps often prevent the timely recognition of everyday risks. In solving this problem, a proactive safety artificial intelligence (AI) system would work bette…
SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories
Chihao Shen, Connor Dilgren, Purva Chiniya +3
This paper introduces SecRepoBench, a benchmark to evaluate code agents on secure code completion in real-world repositories. SecRepoBench has 318 code completion tasks in 27 C/C++…