Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models

Martin Riddell, Ansong Ni, Arman Cohan

While large language models have achieved remarkable performance on various code generation benchmarks, there have been growing concerns regarding potential contamination of these…