Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review

Debalina Ghosh Paul, Hong Zhu, Ian Bayley

With the rapid development of Large Language Models (LLMs), a large number of machine learning models have been developed to assist programming tasks including the generation of pr…