Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

Abstract

Program synthesis has been long studied with recent approaches focused on directly using the power of Large Language Models (LLMs) to generate code. Programming benchmarks, with curated synthesis problems and test-cases, are used to measure the performance of various LLMs on code synthesis. However, these test-cases can be limited in both quantity and quality for fully assessing the functional correctness of the generated code. Such limitation in the existing benchmarks begs the following question: In the era of LLMs, is the code generated really correct? To answer this, we propose EvalPlus -- a code synthesis evaluation framework to rigorously benchmark the functional correctness of LLM-synthesized code. EvalPlus augments a given evaluation dataset with large amounts of test-cases newly produced by an automatic test input generator, powered by both LLM- and mutation-based strategies. While EvalPlus is general, we extend the test-cases of the popular HumanEval benchmark by 80x to build HumanEval+. Our extensive evaluation across 26 popular LLMs (e.g., GPT-4 and ChatGPT) demonstrates that HumanEval+ is able to catch significant amounts of previously undetected wrong code synthesized by LLMs, reducing the pass@k by up-to 19.3-28.9%. We also surprisingly found that test insufficiency can lead to mis-ranking. For example, both WizardCoder-CodeLlama and Phind-CodeLlama now outperform ChatGPT on HumanEval+, while none of them could on HumanEval. Our work not only indicates that prior popular code synthesis evaluation results do not accurately reflect the true performance of LLMs for code synthesis, but also opens up a new direction to improve such programming benchmarks through automated testing. We have open-sourced our tools, enhanced datasets as well as all LLM-generated code at https://github.com/evalplus/evalplus to facilitate and accelerate future LLM-for-code research.

Cited by in corpus (18)

• The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verification
• The Future of AI-Driven Software Engineering
• Measure-Observe-Remeasure: An Interactive Paradigm for Differentially-Private Exploratory Analysis
• Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat
• Automated Data Visualization from Natural Language via Large Language Models: An Exploratory Study
• Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
• Several categories of Large Language Models (LLMs): A Short Survey
• Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency
• Revisiting the Plastic Surgery Hypothesis via Large Language Models
• Isolating Compiler Bugs by Generating Effective Witness Programs with Large Language Models
• LPR: Large Language Models-Aided Program Reduction
• Mokav: Execution-driven Differential Testing with LLMs
• Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models
• ZS4C: Zero-Shot Synthesis of Compilable Code for Incomplete Code Snippets using LLMs
• Exploring the Effectiveness of Abstract Syntax Tree Patterns for Algorithm Recognition
• Let's Take Esoteric Programming Languages Seriously
• TCSR-SQL: Towards Table Content-aware Text-to-SQL with Self-retrieval
• Ethical AI prompt recommendations in large language models using collaborative filtering