11 citations · 11 across the 2 of their papers we have counts for
3 papers
cs.AI2025★ 11 cited
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
Joel Becker, Nate Rush, Elizabeth Barnes +1
Despite widespread adoption, the impact of AI tools on software development in the wild remains understudied. We conduct a randomized controlled trial (RCT) to understand how AI to…
cs.AI2025
HCAST: Human-Calibrated Autonomy Software Tasks
David Rein, Joel Becker, Amy Deng +19
To understand and predict the societal impacts of highly autonomous AI systems, we need benchmarks with grounding, i.e., metrics that directly connect AI performance to real-world…
cs.LG2024
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
Hjalmar Wijk, Tao Lin, Joel Becker +20
Frontier AI safety policies highlight automation of AI research and development (R&D) by AI agents as an important capability to anticipate. However, there exist few evaluations fo…