37 citations · 68 across the 11 of their papers we have counts for
13 papers
PerfBench: Can Agents Resolve Real-World Performance Bugs?
Spandan Garg, Roshanak Zilouchian Moghaddam, Neel Sundaresan
Performance bugs are inefficiencies in software that waste computational resources without causing functional failures, making them particularly challenging to detect and fix. Whil…
When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents
Matous Kozak, Roshanak Zilouchian Moghaddam, Siva Sivaraman
LLM-based coding agents are rapidly being deployed in software development, yet their safety implications remain poorly understood. These agents, while capable of accelerating soft…
FaultLine: Automated Proof-of-Vulnerability Generation Using LLM Agents
Vikram Nitin, Baishakhi Ray, Roshanak Zilouchian Moghaddam
Despite the critical threat posed by software security vulnerabilities, reports are often incomplete, lacking the proof-of-vulnerability (PoV) tests needed to validate fixes and pr…
SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments
Avi Arora, Jinu Jang, Roshanak Zilouchian Moghaddam
Modern Large Language Model (LLM) agents promise end to end assistance with real-world software tasks, yet existing benchmarks evaluate LLM agents almost exclusively in pre-baked e…
The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason
Shanchao Liang, Spandan Garg, Roshanak Zilouchian Moghaddam
As large language models (LLMs) become increasingly capable and widely adopted, benchmarks play a central role in assessing their practical utility. For example, SWE-Bench Verified…
RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code
Dhruv Gautam, Spandan Garg, Jinu Jang +2
Recent advances in language model (LM) agents and function calling have enabled autonomous, feedback-driven systems to solve problems across various digital domains. To better unde…
AutoDev: Automated AI-Driven Development
Michele Tufano, Anisha Agarwal, Jinu Jang +2
The landscape of software development has witnessed a paradigm shift with the advent of AI-powered assistants, exemplified by GitHub Copilot. However, existing solutions are not le…
Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming
Anisha Agarwal, Aaron Chan, Shubham Chandel +6
The integration of Large Language Models (LLMs) into Development Environments (IDEs) has become a focal point in modern software development. LLMs such as OpenAI GPT-3.5/4 and Code…
RAPGen: An Approach for Fixing Code Inefficiencies in Zero-Shot
Spandan Garg, Roshanak Zilouchian Moghaddam, Neel Sundaresan
Performance bugs are non-functional bugs that can even manifest in well-tested commercial products. Fixing these performance bugs is an important yet challenging problem. In this w…
Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?
Aaron Chan, Anant Kharkar, Roshanak Zilouchian Moghaddam +5
Software vulnerabilities bear enterprises significant costs. Despite extensive efforts in research and development of software vulnerability detection methods, uncaught vulnerabili…
DeepPERF: A Deep Learning-Based Approach For Improving Software Performance
Spandan Garg, Roshanak Zilouchian Moghaddam, Colin B. Clement +2
Improving software performance is an important yet challenging part of the software development cycle. Today, the majority of performance inefficiencies are identified and patched…
Generating Examples From CLI Usage: Can Transformers Help?
Roshanak Zilouchian Moghaddam, Spandan Garg, Colin B. Clement +2
Continuous evolution in modern software often causes documentation, tutorials, and examples to be out of sync with changing interfaces and frameworks. Relying on outdated documenta…
Learning to Reduce False Positives in Analytic Bug Detectors
Anant Kharkar, Roshanak Zilouchian Moghaddam, Matthew Jin +4
Due to increasingly complex software design and rapid iterative development, code defects and security vulnerabilities are prevalent in modern software. In response, programmers re…