ContractEval: A Benchmark for Evaluating Contract-Satisfying Assertions in Code Generation

Soohan Lim, Joonghyuk Hahn, Hyunwoo Park +2

Current code generation evaluation measures functional correctness on well-formed inputs that satisfy all input preconditions. This paradigm has a critical limitation: task descrip…