4 papers · 1 filter
Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code
Myeongsoo Kim, Dingmin Wang, Siwei Cui +6
Code agents resolve 65-70% of SWE-bench Verified issues, but Pass@1 cannot tell us why the rest fail, and, as we show, capable-model failures are systematically misdiagnosed withou…
SpecAgent: A Speculative Retrieval and Forecasting Agent for Code Completion
George Ma, Anurag Koul, Qi Chen +6
Large Language Models (LLMs) excel at code-related tasks but often struggle in realistic software repositories, where project-specific APIs and cross-file dependencies are crucial.…
CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance
Myeongsoo Kim, Shweta Garg, Baishakhi Ray +2
Programming assistants powered by large language models have improved dramatically, yet existing benchmarks still evaluate them in narrow code-generation settings. Recent efforts s…
Training Language Model Agents to Find Vulnerabilities with CTF-Dojo
Terry Yue Zhuo, Dingmin Wang, Hantian Ding +2
Large language models (LLMs) have demonstrated exceptional capabilities when trained within executable runtime environments, notably excelling at software engineering tasks through…