46 citations · 104 across the 35 of their papers we have counts for
7 papers · 1 filter
Investigating Test Overfitting on SWE-bench
Toufique Ahmed, Jatin Ganhotra, Avraham Shinnar +1
Tests can be useful towards resolving issues on code repositories. However, relying too much on tests for issue resolution can lead to code that technically passes observed tests b…
Can Old Tests Do New Tricks for Resolving SWE Issues?
Yang Chen, Toufique Ahmed, Reyhaneh Jabbarvand +1
Test suites in real-world projects are often large and achieve high code coverage, yet they remain insufficient for detecting all bugs. The abundance of unresolved issues in open-s…
Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection
Toufique Ahmed, Jatin Ganhotra, Avraham Shinnar +1
A software engineering issue (SWE issue) is easier to resolve when accompanied by a reproduction test. Unfortunately, most issues do not come with functioning reproduction tests, s…
Representing Prompting Patterns with PDL: Compliance Agent Case Study
Mandana Vaziri, Louis Mandel, Yuji Watanabe +3
Prompt engineering for LLMs remains complex, with existing frameworks either hiding complexity behind restrictive APIs or providing inflexible canned patterns that resist customiza…
Improving Examples in Web API Specifications using Iterated-Calls In-Context Learning
Kush Jain, Kiran Kate, Jason Tsay +2
Examples in web API specifications can be essential for API testing, API understanding, and even building chat-bots for APIs. Unfortunately, most API specifications lack human-writ…
AutoPDL: Automatic Prompt Optimization for LLM Agents
Claudio Spiess, Mandana Vaziri, Louis Mandel +1
The performance of large language models (LLMs) depends on how they are prompted, with choices spanning both the high-level prompting pattern (e.g., Zero-Shot, CoT, ReAct, ReWOO) a…
Otter: Generating Tests from Issues to Validate SWE Patches
Toufique Ahmed, Jatin Ganhotra, Rangeet Pan +3
While there has been plenty of work on generating tests from existing code, there has been limited work on generating tests from issues. A correct test must validate the code patch…