1 citations · 2 across the 7 of their papers we have counts for
8 papers
E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch
Hantian Ding, Chloe Bi, Jiacheng Zhu +5
Coding agents powered by large language models (LLMs) are evolving from making localized code changes to developing complete software repositories. However, evaluating repository-s…
Code Monitor Red Teaming for Public-Test-Passing Code
Junchi Liao, Jiawen Deng, Fuji Ren
Visible tests are a common gate for LLM-generated code, but passing them does not certify specification correctness. We study a deployment-like monitoring problem: after code has p…
SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization
Houjun Liu, Lisa Einstein, John Yang +5
LLM coding agents now generate code at an unprecedented scale, yet LLM-generated code introduces cybersecurity vulnerabilities into codebases without human involvement. Even when f…
ProgramBench: Can Language Models Rebuild Programs From Scratch?
John Yang, Kilian Lieret, Jeffrey Ma +9
Turning ideas into full software projects from scratch has become a popular use case for language models. Agents are being deployed to seed, maintain, and grow codebases over exten…
SWE-chat: Coding Agent Interactions From Real Users in the Wild
Joachim Baumann, Vishakh Padmakumar, Xiang Li +3
AI coding agents are being adopted at scale, yet we lack empirical evidence on how people actually use them and how much of their output is useful in practice. We present SWE-chat,…
SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents
Patrick Tser Jern Kon, Archana Pradeep, Ang Chen +5
Small language models (SLMs) offer compelling advantages in cost, latency, and adaptability, but have so far lagged behind larger models on long-horizon software engineering tasks…
OpenThoughts: Data Recipes for Reasoning Models
Etash Guha, Ryan Marten, Sedrick Keh +47
Reasoning models have made rapid progress on many benchmarks involving math, code, and science. Yet, there are still many open questions about the best training recipes for reasoni…
LongCodeBench: Evaluating Coding LLMs at 1M Context Windows
Stefano Rando, Luca Romani, Alessio Sampieri +5
Context lengths for models have grown rapidly, from thousands to millions of tokens in just a few years. The extreme context sizes of modern long-context models have made it diffic…