5 papers
Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code
Myeongsoo Kim, Dingmin Wang, Siwei Cui +6
Code agents resolve 65-70% of SWE-bench Verified issues, but Pass@1 cannot tell us why the rest fail, and, as we show, capable-model failures are systematically misdiagnosed withou…
CODESTRUCT: Code Agents over Structured Action Spaces
Myeongsoo Kim, Joe Hsu, Dingmin Wang +3
LLM-based code agents treat repositories as unstructured text, applying edits through brittle string matching that frequently fails due to formatting drift or ambiguous patterns. W…
CodeScout: Contextual Problem Statement Enhancement for Software Agents
Manan Suri, Xiangci Li, Mehdi Shojaie +5
Current AI-powered code assistance tools often struggle with poorly-defined problem statements that lack sufficient task context and requirements specification. Recent analysis of…
CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance
Myeongsoo Kim, Shweta Garg, Baishakhi Ray +2
Programming assistants powered by large language models have improved dramatically, yet existing benchmarks still evaluate them in narrow code-generation settings. Recent efforts s…
Textual Gradients are a Flawed Metaphor for Automatic Prompt Optimization
Daniel Melcer, Qi Chen, Wen-Hao Chiang +3
A well-engineered prompt can increase the performance of large language models; automatic prompt optimization techniques aim to increase performance without requiring human effort…