DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection

Abstract

There are two main issues in RGB-D salient object detection: (1) how to effectively integrate the complementarity from the cross-modal RGB-D data; (2) how to prevent the contamination effect from the unreliable depth map. In fact, these two problems are linked and intertwined, but the previous methods tend to focus only on the first problem and ignore the consideration of depth map quality, which may yield the model fall into the sub-optimal state. In this paper, we address these two issues in a holistic model synergistically, and propose a novel network named DPANet to explicitly model the potentiality of the depth map and effectively integrate the cross-modal complementarity. By introducing the depth potentiality perception, the network can perceive the potentiality of depth information in a learning-based manner, and guide the fusion process of two modal data to prevent the contamination occurred. The gated multi-modality attention module in the fusion process exploits the attention mechanism with a gate controller to capture long-range dependencies from a cross-modal perspective. Experimental results compared with 15 state-of-the-art methods on 8 datasets demonstrate the validity of the proposed approach both quantitatively and qualitatively.

arXiv commentAccepted by IEEE Transactions on Image Processing 2020

References in corpus (5)

• Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks
• Deeply supervised salient object detection with short connections
• Review of Visual Saliency Detection with Comprehensive Information
• RGBD Salient Object Detection via Deep Fusion
• Co-saliency Detection for RGBD Images Based on Multi-constraint Feature Matching and Cross Label Propagation

Cited by in corpus (14)

• SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection
• RGB-D Salient Object Detection: A Survey
• EDN: Salient Object Detection via Extremely-Downsampled Network
• CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection
• HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection
• Glass Segmentation with RGB-Thermal Image Pairs
• Middle-level Fusion for Lightweight RGB-D Salient Object Detection
• VST++: Efficient and Stronger Visual Saliency Transformer
• Quality-aware Selective Fusion Network for V-D-T Salient Object Detection
• RGB-D Salient Object Detection with Ubiquitous Target Awareness
• An End-to-End Network for Co-Saliency Detection in One Single Image
• Towards Real Zero-Shot Camouflaged Object Segmentation without Camouflaged Annotations
• Modal-Adaptive Gated Recoding Network for RGB-D Salient Object Detection
• RRNet: Relational Reasoning Network with Parallel Multi-scale Attention for Salient Object Detection in Optical Remote Sensing Images