GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks

Abstract

The increasing demand for spatiotemporal data and modeling tasks in geosciences has made geospatial code generation technology a critical factor in enhancing productivity. Although large language models (LLMs) have demonstrated potential in code generation tasks, they often encounter issues such as refusal to code or hallucination in geospatial code generation due to a lack of domain-specific knowledge and code corpora. To address these challenges, this paper presents and open-sources the GeoCode-PT and GeoCode-SFT corpora, along with the GeoCode-Eval evaluation dataset. Additionally, by leveraging QLoRA and LoRA for pretraining and fine-tuning, we introduce GeoCode-GPT-7B, the first LLM focused on geospatial code generation, fine-tuned from Code Llama-7B. Furthermore, we establish a comprehensive geospatial code evaluation framework, incorporating option matching, expert validation, and prompt engineering scoring for LLMs, and systematically evaluate GeoCode-GPT-7B using the GeoCode-Eval dataset. Experimental results show that GeoCode-GPT outperforms other models in multiple-choice accuracy by 9.1% to 32.1%, in code summarization ability by 1.7% to 25.4%, and in code generation capability by 1.2% to 25.1%. This paper provides a solution and empirical validation for enhancing LLMs' performance in geospatial code generation, extends the boundaries of domain-specific model applications, and offers valuable insights into unlocking their potential in geospatial code generation.

References in corpus (20)

• LoRA: Low-Rank Adaptation of Large Language Models
• A Survey of Large Language Models
• Evaluating Large Language Models Trained on Code
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Code Llama: Open Foundation Models for Code
• Large Language Models: A Survey
• Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
• A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models
• DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
• Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture
• WizardCoder: Empowering Code Large Language Models with Evol-Instruct
• Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
• AlpaCare:Instruction-tuned Large Language Models for Medical Application
• Pretrained Language Models for Document-Level Neural Machine Translation
• Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
• A Survey on Evaluating Large Language Models in Code Generation Tasks
• ICE-Score: Instructing Large Language Models to Evaluate Code
• Can Large Language Models Generate Geospatial Code?
• Chain-of-Programming (CoP) : Empowering Large Language Models for Geospatial Code Generation
• GEE-OPs: An Operator Knowledge Base for Geospatial Code Generation on the Google Earth Engine Platform Powered by Large Language Models

Cited by in corpus (3)

• Geo-FuB: A Method for Constructing an Operator-Function Knowledge Base for Geospatial Code Generation Tasks Using Large Language Models
• GeoJSEval: An Automated Evaluation Framework for Large Language Models on JavaScript-Based Geospatial Computation and Visualization Code Generation
• Securing Multi-Agent GIS Systems: Risk Evaluation and Prompt Hardening Optimization