10 citations · 14 across the 5 of their papers we have counts for
3 papers · 1 filter
From Correctness to Collaboration: Toward a Human-Centered Framework for Evaluating AI Agent Behavior in Software Engineering
Tao Dong, Harini Sampath, Ja Young Lee +2
As Large Language Models (LLMs) evolve from code generators into collaborative partners for software engineers, our methods for evaluation are lagging. Current benchmarks, focused…
What do professional software developers need to know to succeed in an age of Artificial Intelligence?
Matthew Kam, Cody Miller, Miaoxin Wang +8
Generative AI is showing early evidence of productivity gains for software developers, but concerns persist regarding workforce disruption and deskilling. We describe our research…
Creating benchmarkable components to measure the quality ofAI-enhanced developer tools
Elise Paradis, Ambar Murillo, Maulishree Pandey +4
In the AI community, benchmarks to evaluate model quality are well established, but an equivalent approach to benchmarking products built upon generative AI models is still missing…