Identical Runs, Different Results: Benchmarking AI Coding Agents on Open-Weight Models

Eduardo Ariño de la Rubia, Szilard Pafka

Repeated runs of the same coding agent are known to give different benchmark scores. We ask what that variation means for a team running an agent on its own task, by intensive repl…