BenchChecker: Assessing the Credibility of Bug-Fixing Benchmarks for LLMs

Di Wu, Xu He, Shu Wang +1

While Large Language Models (LLMs) show promise for Automatic Program Repair (APR), existing benchmarks suffer from data contamination, in which test samples appear in training set…