HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

Abstract

The High-Resolution Transformer (HRFormer) can maintain high-resolution representation and share global receptive fields. It is friendly towards salient object detection (SOD) in which the input and output have the same resolution. However, two critical problems need to be solved for two-modality SOD. One problem is two-modality fusion. The other problem is the HRFormer output's fusion. To address the first problem, a supplementary modality is injected into the primary modality by using global optimization and an attention mechanism to select and purify the modality at the input level. To solve the second problem, a dual-direction short connection fusion module is used to optimize the output features of HRFormer, thereby enhancing the detailed representation of objects at the output level. The proposed model, named HRTransNet, first introduces an auxiliary stream for feature extraction of supplementary modality. Then, features are injected into the primary modality at the beginning of each multi-resolution branch. Next, HRFormer is applied to achieve forwarding propagation. Finally, all the output features with different resolutions are aggregated by intra-feature and inter-feature interactive transformers. Application of the proposed model results in impressive improvement for driving two-modality SOD tasks, e.g., RGB-D, RGB-T, and light field SOD.https://github.com/liuzywen/HRTransNet

References in corpus (17)

• Salient Object Detection: A Benchmark
• Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks
• High-Resolution Representations for Labeling Pixels and Regions
• Review of Visual Saliency Detection with Comprehensive Information
• SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection
• Multi-interactive Dual-decoder for RGB-thermal Salient Object Detection
• DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection
• TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network
• Light Field Salient Object Detection: A Review and Benchmark
• SAC-Net: Spatial Attenuation Context for Salient Object Detection
• RGB-D Salient Object Detection with Ubiquitous Target Awareness
• Generative Transformer for Accurate and Reliable Salient Object Detection
• Unifying Global-Local Representations in Salient Object Detection with Transformer
• CoSformer: Detecting Co-Salient Object with Transformers
• Gaze Estimation with an Ensemble of Four Architectures
• Transformer-based Network for RGB-D Saliency Detection
• Dense Prediction with Attentive Feature Aggregation

Cited by in corpus (4)

• VST++: Efficient and Stronger Visual Saliency Transformer
• Quality-aware Selective Fusion Network for V-D-T Salient Object Detection
• Deep Fourier-embedded Network for RGB and Thermal Salient Object Detection
• Efficient Fourier Filtering Network with Contrastive Learning for AAV-based Unaligned Bimodal Salient Object Detection