Co-Salient Object Detection with Semantic-Level Consensus Extraction and Dispersion

Abstract

Given a group of images, co-salient object detection (CoSOD) aims to highlight the common salient object in each image. There are two factors closely related to the success of this task, namely consensus extraction, and the dispersion of consensus to each image. Most previous works represent the group consensus using local features, while we instead utilize a hierarchical Transformer module for extracting semantic-level consensus. Therefore, it can obtain a more comprehensive representation of the common object category, and exclude interference from other objects that share local similarities with the target object. In addition, we propose a Transformer-based dispersion module that takes into account the variation of the co-salient object in different scenes. It distributes the consensus to the image feature maps in an image-specific way while making full use of interactions within the group. These two modules are integrated with a ViT encoder and an FPN-like decoder to form an end-to-end trainable network, without additional branch and auxiliary loss. The proposed method is evaluated on three commonly used CoSOD datasets and achieves state-of-the-art performance.

arXiv commentAccepted by ACM MM 2023

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• PVT v2: Improved Baselines with Pyramid Vision Transformer
• BEiT: BERT Pre-Training of Image Transformers
• Vision Transformer Adapter for Dense Predictions
• TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network
• Re-thinking Co-Salient Object Detection
• GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector