416 citations · 698 across the 23 of their papers we have counts for
17 papers
PerfBench: Can Agents Resolve Real-World Performance Bugs?
Spandan Garg, Roshanak Zilouchian Moghaddam, Neel Sundaresan
Performance bugs are inefficiencies in software that waste computational resources without causing functional failures, making them particularly challenging to detect and fix. Whil…
RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code
Dhruv Gautam, Spandan Garg, Jinu Jang +2
Recent advances in language model (LM) agents and function calling have enabled autonomous, feedback-driven systems to solve problems across various digital domains. To better unde…
Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation
Benjamin Steenhoek, Michele Tufano, Neel Sundaresan +1
Software testing is a crucial but time-consuming aspect of software development, and recently, Large Language Models (LLMs) have gained popularity for automated test case generatio…
Exploring and Evaluating Personalized Models for Code Generation
Andrei Zlotchevski, Dawn Drain, Alexey Svyatkovskiy +3
Large Transformer models achieved the state-of-the-art status for Natural Language Understanding tasks and are increasingly becoming the baseline model architecture for modeling so…
Generating Examples From CLI Usage: Can Transformers Help?
Roshanak Zilouchian Moghaddam, Spandan Garg, Colin B. Clement +2
Continuous evolution in modern software often causes documentation, tutorials, and examples to be out of sync with changing interfaces and frameworks. Relying on outdated documenta…
Methods2Test: A dataset of focal methods mapped to test cases
Michele Tufano, Shao Kun Deng, Neel Sundaresan +1
Unit testing is an essential part of the software development process, which helps to identify issues with source code in early stages of development and prevent regressions. Machi…