3 citations · 7 across the 16 of their papers we have counts for
8 papers · 1 filter
Shortcutting the Fix: Identifying and Categorizing Agentic Exploits in Software Engineering Benchmarks
Nikolai Ludwig, Wasi Uddin Ahmad, Somshubra Majumdar +1
While autonomous software engineering (SWE) agents achieve high benchmark resolution rates, these scores can mask exploitative behaviors---such as leveraging local Git histories, a…
Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents
Wasi Uddin Ahmad, Nikolai Ludwig, Somshubra Majumdar +1
The path toward autonomous software engineering is currently bottlenecked by a severe deficit of diverse, large-scale trajectory data. We address this by introducing \ourdataset, a…
From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agents
Nikolai Ludwig, Wasi Uddin Ahmad, Somshubra Majumdar +1
We introduce SWE-ZERO to SWE-HERO, a two-stage SFT recipe that achieves state-of-the-art results on SWE-bench by distilling open-weight frontier LLMs. Our pipeline replaces resourc…
BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
Terry Yue Zhuo, Xiaolong Jin, Hange Liu +37
Crowdsourced model evaluation platforms, such as Chatbot Arena, enable real-time evaluation from human perspectives to assess the quality of model responses. In the coding domain,…
From Output to Evaluation: Does Raw Instruction-Tuned Code LLMs Output Suffice for Fill-in-the-Middle Code Generation?
Wasi Uddin Ahmad, Somshubra Majumdar, Boris Ginsburg
Post-processing is crucial for the automatic evaluation of LLMs in fill-in-the-middle (FIM) code generation due to the frequent presence of extraneous code in raw outputs. This ext…
OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs
Wasi Uddin Ahmad, Aleksander Ficek, Mehrzad Samadi +4
Large Language Models (LLMs) have transformed software development by enabling code generation, automated debugging, and complex reasoning. However, their continued advancement is…