4 papers
Improving Examples in Web API Specifications using Iterated-Calls In-Context Learning
Kush Jain, Kiran Kate, Jason Tsay +2
Examples in web API specifications can be essential for API testing, API understanding, and even building chat-bots for APIs. Unfortunately, most API specifications lack human-writ…
Are Large Language Models Memorizing Bug Benchmarks?
Daniel Ramos, Claudia Mamede, Kush Jain +3
Large Language Models (LLMs) have become integral to various software engineering tasks, including code generation, bug detection, and repair. To evaluate model performance in thes…
TestForge: Feedback-Driven, Agentic Test Suite Generation
Kush Jain, Claire Le Goues
Automated test generation holds great promise for alleviating the burdens of manual test creation. However, existing search-based techniques compromise on test readability, while L…
TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark
Kush Jain, Gabriel Synnaeve, Baptiste Rozière
Code generation models can help improve many common software tasks ranging from code completion to defect prediction. Most of the existing benchmarks for code generation LLMs focus…