2 citations · 5 across the 7 of their papers we have counts for
4 papers · 1 filter
SWE-Bench Pro V2: A Cleaner, Harder-to-Game Coding Benchmark
Soham Dan, Jeff Da, Sanyam Bhutani +8
SWE-Bench Pro is a widely used benchmark of long-horizon software engineering tasks for coding agents. As agents improve, they reach task issues earlier agents never touched and in…
Maintaining Benchmarks Against Increasingly Capable Agents: Detection and Remediation of Unearned Passes
Weijun Luo, Kelvin Luu, Xinyi Liu +6
Agentic benchmarks guide model selection and training. Yet an agent can pass a task without demonstrating the intended capability. Such outcomes constitute unearned passes; their p…
SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution
Mohit Raghavendra, Soham Dan, Miguel Romero Calvo +12
We introduce SWE Atlas, a benchmark suite for coding agents spanning three professional software engineering workflows: Codebase Q&A (124 tasks), Test Writing (90 tasks), and Refac…
Improving Robustness of Tabular Retrieval via Representational Stability
Kushal Raj Bhandari, Adarsh Singh, Jianxi Gao +2
Transformer-based table retrieval systems flatten structured tables into token sequences, making retrieval sensitive to the choice of serialization even when table semantics remain…