416 citations · 915 across the 36 of their papers we have counts for
37 papers
PerfBench: Can Agents Resolve Real-World Performance Bugs?
Spandan Garg, Roshanak Zilouchian Moghaddam, Neel Sundaresan
Performance bugs are inefficiencies in software that waste computational resources without causing functional failures, making them particularly challenging to detect and fix. Whil…
RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code
Dhruv Gautam, Spandan Garg, Jinu Jang +2
Recent advances in language model (LM) agents and function calling have enabled autonomous, feedback-driven systems to solve problems across various digital domains. To better unde…
Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation
Benjamin Steenhoek, Michele Tufano, Neel Sundaresan +1
Software testing is a crucial but time-consuming aspect of software development, and recently, Large Language Models (LLMs) have gained popularity for automated test case generatio…
Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension
Mengnan Qi, Yufan Huang, Yongqiang Yao +3
Large language models (LLMs) has experienced exponential growth, they demonstrate remarkable performance across various tasks. Notwithstanding, contemporary research primarily cent…
AutoDev: Automated AI-Driven Development
Michele Tufano, Anisha Agarwal, Jinu Jang +2
The landscape of software development has witnessed a paradigm shift with the advent of AI-powered assistants, exemplified by GitHub Copilot. However, existing solutions are not le…
Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming
Anisha Agarwal, Aaron Chan, Shubham Chandel +6
The integration of Large Language Models (LLMs) into Development Environments (IDEs) has become a focal point in modern software development. LLMs such as OpenAI GPT-3.5/4 and Code…
Rethinking the Instruction Quality: LIFT is What You Need
Yang Xu, Yongqiang Yao, Yufan Huang +4
Instruction tuning, a specialized technique to enhance large language model (LLM) performance via instruction datasets, relies heavily on the quality of employed data. Existing qua…
SUT: Active Defects Probing for Transcompiler Models
Mengnan Qi, Yufan Huang, Maoquan Wang +5
Automatic Program translation has enormous application value and hence has been attracting significant interest from AI researchers. However, we observe that current program transl…
Program Translation via Code Distillation
Yufan Huang, Mengnan Qi, Yongqiang Yao +4
Software version migration and program translation are an important and costly part of the lifecycle of large codebases. Traditional machine translation relies on parallel corpora…
Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation
Benjamin Steenhoek, Michele Tufano, Neel Sundaresan +1
Software testing is a crucial aspect of software development, and the creation of high-quality tests that adhere to best practices is essential for effective maintenance. Recently,…
Predicting Code Coverage without Execution
Michele Tufano, Shubham Chandel, Anisha Agarwal +2
Code coverage is a widely used metric for quantifying the extent to which program elements, such as statements or branches, are executed during testing. Calculating code coverage i…
RAPGen: An Approach for Fixing Code Inefficiencies in Zero-Shot
Spandan Garg, Roshanak Zilouchian Moghaddam, Neel Sundaresan
Performance bugs are non-functional bugs that can even manifest in well-tested commercial products. Fixing these performance bugs is an important yet challenging problem. In this w…
Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?
Aaron Chan, Anant Kharkar, Roshanak Zilouchian Moghaddam +5
Software vulnerabilities bear enterprises significant costs. Despite extensive efforts in research and development of software vulnerability detection methods, uncaught vulnerabili…
Code Execution with Pre-trained Language Models
Chenxiao Liu, Shuai Lu, Weizhu Chen +5
Code execution is a fundamental aspect of programming language semantics that reflects the exact behavior of the code. However, most pre-trained models for code intelligence ignore…
InferFix: End-to-End Program Repair with LLMs
Matthew Jin, Syed Shahriar, Michele Tufano +4
Software development life cycle is profoundly influenced by bugs: their introduction, identification, and eventual resolution account for a significant portion of software cost. Th…
Exploring and Evaluating Personalized Models for Code Generation
Andrei Zlotchevski, Dawn Drain, Alexey Svyatkovskiy +3
Large Transformer models achieved the state-of-the-art status for Natural Language Understanding tasks and are increasingly becoming the baseline model architecture for modeling so…
DeepPERF: A Deep Learning-Based Approach For Improving Software Performance
Spandan Garg, Roshanak Zilouchian Moghaddam, Colin B. Clement +2
Improving software performance is an important yet challenging part of the software development cycle. Today, the majority of performance inefficiencies are identified and patched…
AdaptivePaste: Code Adaptation through Learning Semantics-aware Variable Usage Representations
Xiaoyu Liu, Jinu Jang, Neel Sundaresan +2
In software development, it is common for programmers to copy-paste or port code snippets and then adapt them to their use case. This scenario motivates the code adaptation task --…
Generating Examples From CLI Usage: Can Transformers Help?
Roshanak Zilouchian Moghaddam, Spandan Garg, Colin B. Clement +2
Continuous evolution in modern software often causes documentation, tutorials, and examples to be out of sync with changing interfaces and frameworks. Relying on outdated documenta…
Methods2Test: A dataset of focal methods mapped to test cases
Michele Tufano, Shao Kun Deng, Neel Sundaresan +1
Unit testing is an essential part of the software development process, which helps to identify issues with source code in early stages of development and prevent regressions. Machi…
Learning to Reduce False Positives in Analytic Bug Detectors
Anant Kharkar, Roshanak Zilouchian Moghaddam, Matthew Jin +4
Due to increasingly complex software design and rapid iterative development, code defects and security vulnerabilities are prevalent in modern software. In response, programmers re…
Automating Code Review Activities by Large-Scale Pre-training
Zhiyu Li, Shuai Lu, Daya Guo +8
Code review is an essential part to software development lifecycle since it aims at guaranteeing the quality of codes. Modern code review activities necessitate developers viewing,…
Training and Evaluating a Jupyter Notebook Data Science Assistant
Shubham Chandel, Colin B. Clement, Guillermo Serrato +1
We study the feasibility of a Data Science assistant powered by a sequence-to-sequence transformer by training a new model JuPyT5 on all publicly available Jupyter Notebook GitHub…
Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchy
Colin B. Clement, Shuai Lu, Xiaoyu Liu +5
Statistical language modeling and translation with transformers have found many successful applications in program understanding and generation tasks, setting high benchmarks for t…