Salient Object Detection in Optical Remote Sensing Images Driven by Transformer
arXiv:2309.08206 · doi:10.1109/TIP.2023.3314285
Abstract
Existing methods for Salient Object Detection in Optical Remote Sensing Images (ORSI-SOD) mainly adopt Convolutional Neural Networks (CNNs) as the backbone, such as VGG and ResNet. Since CNNs can only extract features within certain receptive fields, most ORSI-SOD methods generally follow the local-to-contextual paradigm. In this paper, we propose a novel Global Extraction Local Exploration Network (GeleNet) for ORSI-SOD following the global-to-local paradigm. Specifically, GeleNet first adopts a transformer backbone to generate four-level feature embeddings with global long-range dependencies. Then, GeleNet employs a Direction-aware Shuffle Weighted Spatial Attention Module (D-SWSAM) and its simplified version (SWSAM) to enhance local interactions, and a Knowledge Transfer Module (KTM) to further enhance cross-level contextual interactions. D-SWSAM comprehensively perceives the orientation information in the lowest-level features through directional convolutions to adapt to various orientations of salient objects in ORSIs, and effectively enhances the details of salient objects with an improved attention mechanism. SWSAM discards the direction-aware part of D-SWSAM to focus on localizing salient objects in the highest-level features. KTM models the contextual correlation knowledge of two middle-level features of different scales based on the self-attention mechanism, and transfers the knowledge to the raw features to generate more discriminative features. Finally, a saliency predictor is used to generate the saliency map based on the outputs of the above three modules. Extensive experiments on three public datasets demonstrate that the proposed GeleNet outperforms relevant state-of-the-art methods. The code and results of our method are available at https://github.com/MathLee/GeleNet.
arXiv comment13 pages, 6 figures, Accepted by IEEE Transactions on Image Processing 2023
References in corpus (21)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- PVT v2: Improved Baselines with Pyramid Vision Transformer
- Squeeze-and-Excitation Networks
- Salient Object Detection: A Benchmark
- Review of Visual Saliency Detection with Comprehensive Information
- SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection
- Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images
- Nested Network with Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images
- RGB-T Semantic Segmentation with Location, Activation, and Sharpening
- Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks
- TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network
- Lightweight Salient Object Detection in Optical Remote Sensing Images via Feature Correlation
- Lightweight Salient Object Detection in Optical Remote-Sensing Images via Semantic Matching and Edge Alignment
- Multi-Content Complementation Network for Salient Object Detection in Optical Remote Sensing Images
- A Parallel Down-Up Fusion Network for Salient Object Detection in Optical Remote Sensing Images
- Personal Fixations-Based Object Segmentation with Object Localization and Boundary Preservation
- Generative Transformer for Accurate and Reliable Salient Object Detection
- Salient Object Detection via Integrity Learning
- Densely Nested Top-Down Flows for Salient Object Detection