CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection

Abstract

Focusing on the issue of how to effectively capture and utilize cross-modality information in RGB-D salient object detection (SOD) task, we present a convolutional neural network (CNN) model, named CIR-Net, based on the novel cross-modality interaction and refinement. For the cross-modality interaction, 1) a progressive attention guided integration unit is proposed to sufficiently integrate RGB-D feature representations in the encoder stage, and 2) a convergence aggregation structure is proposed, which flows the RGB and depth decoding features into the corresponding RGB-D decoding streams via an importance gated fusion unit in the decoder stage. For the cross-modality refinement, we insert a refinement middleware structure between the encoder and the decoder, in which the RGB, depth, and RGB-D encoder features are further refined by successively using a self-modality attention refinement unit and a cross-modality weighting refinement unit. At last, with the gradually refined features, we predict the saliency map in the decoder stage. Extensive experiments on six popular RGB-D SOD benchmarks demonstrate that our network outperforms the state-of-the-art saliency detectors both qualitatively and quantitatively.

arXiv commentAccepted by IEEE Transactions on Image Processing 2022, 16 pages, 11 figures

References in corpus (11)

• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks
• Deeply supervised salient object detection with short connections
• Salient Object Detection: A Survey
• Review of Visual Saliency Detection with Comprehensive Information
• RGBD Salient Object Detection via Deep Fusion
• DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection
• Bilateral Attention Network for RGB-D Salient Object Detection
• Co-saliency Detection for RGBD Images Based on Multi-constraint Feature Matching and Cross Label Propagation
• Depth Quality Aware Salient Object Detection
• Personal Fixations-Based Object Segmentation with Object Localization and Boundary Preservation

Cited by in corpus (6)

• Guided Depth Map Super-resolution: A Survey
• Quality-aware Selective Fusion Network for V-D-T Salient Object Detection
• Depth Awakens: A Depth-perceptual Attention Fusion Network for RGB-D Camouflaged Object Detection
• ViDSOD-100: A New Dataset and a Baseline Model for RGB-D Video Salient Object Detection
• Efficient Fourier Filtering Network with Contrastive Learning for AAV-based Unaligned Bimodal Salient Object Detection
• LEAF-Mamba: Local Emphatic and Adaptive Fusion State Space Model for RGB-D Salient Object Detection