A Task-Level Evaluation of AI Agents in Open-Source Projects

Shojibur Rahman, Md Fazle Rabbi, Minhaz Zibran

In this paper, we present a comparative study of five autonomous coding agents using AIDev-pop, which is a public dataset containing thousands of AI-generated pull requests (PRs) a…