Can Language Models Replace Programmers for Coding? REPOCOD Says 'Not Yet'

Shanchao Liang, Yiran Hu, Nan Jiang +1

Recently, a number of repository-level code generation benchmarks-such as CoderEval, DevEval, RepoEval, RepoBench, and LongCodeArena-have emerged to evaluate the capabilities of la…