NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts

Shudan Zhang, Hanlin Zhao, Xiao Liu +6

Large language models (LLMs) have manifested strong ability to generate codes for productive activities. However, current benchmarks for code synthesis, such as HumanEval, MBPP, an…