4 papers
ContractEval: A Benchmark for Evaluating Contract-Satisfying Assertions in Code Generation
Soohan Lim, Joonghyuk Hahn, Hyunwoo Park +2
Current code generation evaluation measures functional correctness on well-formed inputs that satisfy all input preconditions. This paradigm has a critical limitation: task descrip…
SAGE:Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMs
Aditi, Hyunwoo Park, Sicheol Sung +2
Grammar-based test case generation has proven effective for competitive programming problems, but generating valid and general grammars from natural language specifications remains…
LogiCase: Effective Test Case Generation from Logical Description in Competitive Programming
Sicheol Sung, Aditi, Dogyu kim +2
Automated Test Case Generation (ATCG) is crucial for evaluating software reliability, particularly in competitive programming where robust algorithm assessments depend on diverse a…
CodeComplex: Dataset for Worst-Case Time Complexity Prediction
Seung-Yeop Baik, Joonghyuk Hahn, Jungin Kim +4
Reasoning ability of Large Language Models (LLMs) is a crucial ability, especially in complex decision-making tasks. One significant task to show LLMs' reasoning capability is code…