A Comprehensive Review of State-of-The-Art Methods for Java Code Generation from Natural Language Text
arXiv:2306.06371 · doi:10.1016/j.nlp.2023.100013
Abstract
Java Code Generation consists in generating automatically Java code from a Natural Language Text. This NLP task helps in increasing programmers' productivity by providing them with immediate solutions to the simplest and most repetitive tasks. Code generation is a challenging task because of the hard syntactic rules and the necessity of a deep understanding of the semantic aspect of the programming language. Many works tried to tackle this task using either RNN-based, or Transformer-based models. The latter achieved remarkable advancement in the domain and they can be divided into three groups: (1) encoder-only models, (2) decoder-only models, and (3) encoder-decoder models. In this paper, we provide a comprehensive review of the evolution and progress of deep learning models in Java code generation task. We focus on the most important methods and present their merits and limitations, as well as the objective functions used by the community. In addition, we provide a detailed description of datasets and evaluation metrics used in the literature. Finally, we discuss results of different models on CONCODE dataset, then propose some future directions.
Published at Elsevier's Natural Language Processing Journal
References in corpus (14)
- Training language models to follow instructions with human feedback
- LLaMA: Open and Efficient Foundation Language Models
- Evaluating Large Language Models Trained on Code
- Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars
- LaMDA: Language Models for Dialog Applications
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
- PaLM-E: An Embodied Multimodal Language Model
- Scaling Language Models: Methods, Analysis & Insights from Training Gopher
- CodeBLEU: a Method for Automatic Evaluation of Code Synthesis
- Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
- Root Mean Square Layer Normalization
- CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning
- The Stack: 3 TB of permissively licensed source code
- JaCoText: A Pretrained Model for Java Code-Text Generation