DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
arXiv:2401.14196
Abstract
The rapid development of large language models has revolutionized code intelligence in software development. However, the predominance of closed-source models has restricted extensive research and development. To address this, we introduce the DeepSeek-Coder series, a range of open-source code models with sizes from 1.3B to 33B, trained from scratch on 2 trillion tokens. These models are pre-trained on a high-quality project-level code corpus and employ a fill-in-the-blank task with a 16K window to enhance code generation and infilling. Our extensive evaluations demonstrate that DeepSeek-Coder not only achieves state-of-the-art performance among open-source code models across multiple benchmarks but also surpasses existing closed-source models like Codex and GPT-3.5. Furthermore, DeepSeek-Coder models are under a permissive license that allows for both research and unrestricted commercial use.
Cited by in corpus (9)
- RTLCoder: Outperforming GPT-3.5 in Design RTL Generation with Our Open-Source Dataset and Lightweight Solution
- UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility
- The Dawn of Natural Language to SQL: Are We Fully Ready?
- A Comprehensive Survey on Integrating Large Language Models with Knowledge-Based Methods
- GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks
- CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models
- Type-Constrained Code Generation with Language Models
- Generating Energy-efficient code with LLMs
- AI-based Attacker Models for Enhancing Multi-Stage Cyberattack Simulations in Smart Grids Using Co-Simulation Environments