Diff-Based Code Corruption using LLMs for Large-Scale Bugfix Benchmarking

Balázs Szalontai, Ábel Szauter, Balázs Márton +3

There are various benchmarks to evaluate bugfixing capabilities of Large Language Models. However, most widespread benchmarks do not fully reflect real-world bugfixing practices. T…