3 papers
cs.LG2026
On Randomness in Agentic Evals
Bjarni Haukur Bjarnason, André Silva, Martin Monperrus
Agentic systems are evaluated on benchmarks where agents interact with environments to solve tasks. Most papers report a pass@1 score computed from a single run per task, assuming…
cs.PL2025
Gradient-Based Program Repair: Fixing Bugs in Continuous Program Spaces
André Silva, Gustav Thorén, Martin Monperrus
Automatic program repair seeks to generate correct code from buggy programs, with most approaches searching the correct program in a discrete, symbolic space of source code tokens.…
cs.SE2024
RepairBench: Leaderboard of Frontier Models for Program Repair
André Silva, Martin Monperrus
AI-driven program repair uses AI models to repair buggy software by producing patches. Rapid advancements in AI surely impact state-of-the-art performance of program repair. Yet, g…