WizardCoder: Empowering Code Large Language Models with Evol-Instruct

Abstract

Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM

Large Language model, Code Generation, Code LLMs.This paper has been accepted to ICLR 2024. Please cite the ICLR version

Cited by in corpus (14)

• ChatEDA: A Large Language Model Powered Autonomous Agent for EDA
• LLM for SoC Security: A Paradigm Shift
• Refactoring Programs Using Large Language Models with Few-Shot Examples
• Synthetic Data Generation Using Large Language Models: Advances in Text and Code
• GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks
• Several categories of Large Language Models (LLMs): A Short Survey
• CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models
• Output Format Biases in the Evaluation of Large Language Models for Code Translation
• SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework
• Metamorphic Malware Evolution: The Potential and Peril of Large Language Models
• Detect Llama -- Finding Vulnerabilities in Smart Contracts using Large Language Models
• Program Repair with Minimal Edits Using CodeT5
• LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codes
• Trained Without My Consent: Detecting Code Inclusion In Language Models Trained on Code