Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysis

Abstract

Monitoring changes in the Earth's surface is crucial for understanding natural processes and human impacts, necessitating precise and comprehensive interpretation methodologies. Remote sensing satellite imagery offers a unique perspective for monitoring these changes, leading to the emergence of remote sensing image change interpretation (RSICI) as a significant research focus. Current RSICI technology encompasses change detection and change captioning, each with its limitations in providing comprehensive interpretation. To address this, we propose an interactive Change-Agent, which can follow user instructions to achieve comprehensive change interpretation and insightful analysis, such as change detection and change captioning, change object counting, change cause analysis, etc. The Change-Agent integrates a multi-level change interpretation (MCI) model as the eyes and a large language model (LLM) as the brain. The MCI model contains two branches of pixel-level change detection and semantic-level change captioning, in which the BI-temporal Iterative Interaction (BI3) layer is proposed to enhance the model's discriminative feature representation capabilities. To support the training of the MCI model, we build the LEVIR-MCI dataset with a large number of change masks and captions of changes. Experiments demonstrate the SOTA performance of the MCI model in achieving both change detection and change description simultaneously, and highlight the promising application value of our Change-Agent in facilitating comprehensive interpretation of surface changes, which opens up a new avenue for intelligent remote sensing applications. To facilitate future research, we will make our dataset and codebase of the MCI model and Change-Agent publicly available at https://github.com/Chen-Yang-Liu/Change-Agent

IEEE Transactions on Geoscience and Remote Sensing 2024

References in corpus (9)

• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Remote Sensing Image Change Detection with Transformers
• ReAct: Synergizing Reasoning and Acting in Language Models
• MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
• Joint Spatio-Temporal Modeling for the Semantic Change Detection in Remote Sensing Images
• Changes to Captions: An Attentive Network for Remote Sensing Change Captioning
• Implicit Ray-Transformers for Multi-view Remote Sensing Image Segmentation
• Dense Pixel-to-Pixel Harmonization via Continuous Image Representation
• ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models

Cited by in corpus (6)

• Remote Sensing SpatioTemporal Vision-Language Models: A Comprehensive Survey
• When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
• Mask Approximation Net: A Novel Diffusion Model Approach for Remote Sensing Change Captioning
• Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Framework
• SeG-SR: Integrating Semantic Knowledge into Remote Sensing Image Super-Resolution via Vision-Language Model
• MMUEChange: A Generalized LLM Agent Framework for Intelligent Multi-Modal Urban Environment Change Analysis