Execution-Based Evaluation for Open-Domain Code Generation

Zhiruo Wang, Shuyan Zhou, Daniel Fried +1

To extend the scope of coding queries to more realistic settings, we propose ODEX, the first Open-Domain EXecution-based natural language (NL) to Python code generation dataset. OD…