Data-Driven Vision-Language Models for Remote Sensing: A Survey

Abstract

During the deep learning era, innovations in remote sensing (RS) vision models primarily focused on optimizing network architectures for specific tasks and conducting end-to-end training. However, with the advent of the Vision-Language Models (VLMs) era, constrained by substantial computational resource demands, the mainstream research paradigm has shifted towards adapting general-purpose VLMs using RS vertical domain data for fine-tuning, rather than modifying their core architectures. This transition marks a change in the innovation pathway from model-centric to data-centric. Despite the increasing importance of data, the academic community still lacks a systematic, data-centric survey to guide researchers in constructing high-quality RS vision-language datasets for various application scenarios and in conducting effective model fine-tuning and performance evaluation based on these datasets. To address this gap, this paper presents a systematic review of the development status of RS datasets tailored for VLMs. It further provides a comprehensive practical guide for RS researchers in the large model era, covering the entire pipeline from data construction to fine-tuning and evaluation. This paper aims to encourage the academic community to build more diverse and higher-quality vision-language datasets for specialized RS sub-fields. Ultimately, the goal is to converge these efforts into a comprehensive RS vision-language corpus that offers extensive task coverage and a wide geographical distribution, thereby laying a solid data foundation for advancing towards general artificial intelligence in RS.

arXiv comment44 pages, 15 figures

References in corpus (21)

• Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark
• Exploring Models and Data for Remote Sensing Image Caption Generation
• MMRotate: A Rotated Object Detection Benchmark using PyTorch
• RSVQA: Visual Question Answering for Remote Sensing Data
• VLP: A Survey on Vision-Language Pre-training
• Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrieval
• A Review of Location Encoding for GeoAI: Methods and Applications
• RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data
• RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing
• Deep multi-task learning for a geographically-regularized semantic segmentation of aerial images
• Learning from models beyond fine-tuning
• Change Detection Meets Visual Question Answering
• When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
• Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives
• A Review of Emerging Research Directions in Abstract Visual Reasoning
• An overview of domain-specific foundation model: key technologies, applications and challenges
• UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces
• RSTeller: Scaling Up Visual Language Modeling in Remote Sensing with Rich Linguistic Semantics from Openly Available Data and Large Language Models
• ImageRAG: Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAG
• CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
• Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuning