Out of the BLEU: how should we assess quality of the Code Generation models?

Mikhail Evtikhiev, Egor Bogomolov, Yaroslav Sokolov +1

In recent years, researchers have created and introduced a significant number of various code generation models. As human evaluation of every new model version is unfeasible, the c…