CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models

Hao Yu, Bo Shen, Dezhi Ran +7

Code generation models based on the pre-training and fine-tuning paradigm have been increasingly attempted by both academia and industry, resulting in well-known industrial models…