Multi-stage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Images

Abstract

The attention mechanism can refine the extracted feature maps and boost the classification performance of the deep network, which has become an essential technique in computer vision and natural language processing. However, the memory and computational costs of the dot-product attention mechanism increase quadratically with the spatio-temporal size of the input. Such growth hinders the usage of attention mechanisms considerably in application scenarios with large-scale inputs. In this Letter, we propose a Linear Attention Mechanism (LAM) to address this issue, which is approximately equivalent to dot-product attention with computational efficiency. Such a design makes the incorporation between attention mechanisms and deep networks much more flexible and versatile. Based on the proposed LAM, we re-factor the skip connections in the raw U-Net and design a Multi-stage Attention ResU-Net (MAResU-Net) for semantic segmentation from fine-resolution remote sensing images. Experiments conducted on the Vaihingen dataset demonstrated the effectiveness and efficiency of our MAResU-Net. Open-source code is available at https://github.com/lironui/Multistage-Attention-ResU-Net.

arXiv commentarXiv admin note: substantial text overlap with arXiv:2007.14902, arXiv:2009.02130

References in corpus (11)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• CE-Net: Context Encoder Network for 2D Medical Image Segmentation
• Squeeze-and-Excitation Networks
• ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data
• OCNet: Object Context Network for Scene Parsing
• Generating Long Sequences with Sparse Transformers
• Reformer: The Efficient Transformer
• Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
• MACU-Net for Semantic Segmentation of Fine-Resolution Remotely Sensed Images
• Efficient Attention: Attention with Linear Complexities

Cited by in corpus (14)

• UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery
• ABCNet: Attentive Bilateral Contextual Network for Efficient Semantic Segmentation of Fine-Resolution Remote Sensing Images
• A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing Images
• Multi-Attention-Network for Semantic Segmentation of Fine Resolution Remote Sensing Images
• A2-FPN for Semantic Segmentation of Fine-Resolution Remotely Sensed Images
• Building extraction with vision transformer
• Land Cover Classification from Remote Sensing Images Based on Multi-Scale Fully Convolutional Network
• A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation
• Imbalance Knowledge-Driven Multi-modal Network for Land-Cover Semantic Segmentation Using Images and LiDAR Point Clouds
• Semantic Labeling of High Resolution Images Using EfficientUNets and Transformers
• Implicit Ray-Transformers for Multi-view Remote Sensing Image Segmentation
• Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images
• AeroReformer: Aerial Referring Transformer for UAV-based Referring Image Segmentation
• Multi-Head Linear Attention Generative Adversarial Network for Thin Cloud Removal