Re-thinking Co-Salient Object Detection

Abstract

In this paper, we conduct a comprehensive study on the co-salient object detection (CoSOD) problem for images. CoSOD is an emerging and rapidly growing extension of salient object detection (SOD), which aims to detect the co-occurring salient objects in a group of images. However, existing CoSOD datasets often have a serious data bias, assuming that each group of images contains salient objects of similar visual appearances. This bias can lead to the ideal settings and effectiveness of models trained on existing datasets, being impaired in real-life situations, where similarities are usually semantic or conceptual. To tackle this issue, we first introduce a new benchmark, called CoSOD3k in the wild, which requires a large amount of semantic context, making it more challenging than existing CoSOD datasets. Our CoSOD3k consists of 3,316 high-quality, elaborately selected images divided into 160 groups with hierarchical annotations. The images span a wide range of categories, shapes, object sizes, and backgrounds. Second, we integrate the existing SOD techniques to build a unified, trainable CoSOD framework, which is long overdue in this field. Specifically, we propose a novel CoEG-Net that augments our prior model EGNet with a co-attention projection strategy to enable fast common information learning. CoEG-Net fully leverages previous large-scale SOD datasets and significantly improves the model scalability and stability. Third, we comprehensively summarize 40 cutting-edge algorithms, benchmarking 18 of them over three challenging CoSOD datasets (iCoSeg, CoSal2015, and our CoSOD3k), and reporting more detailed (i.e., group-level) performance analysis. Finally, we discuss the challenges and future works of CoSOD. We hope that our study will give a strong boost to growth in the CoSOD community. The benchmark toolbox and results are available on our project page at http://dpfan.net/CoSOD3K/.

arXiv comment22pages, 18 figures. CVPR2020-CoSOD3K extension. Code: https://github.com/DengPingFan/CoEGNet

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• Res2Net: A New Multi-scale Backbone Architecture
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Salient Object Detection: A Benchmark
• Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks
• Salient Object Detection: A Survey
• Review of Visual Saliency Detection with Comprehensive Information
• RGB-D Salient Object Detection: A Survey
• Nested Network with Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images
• Salient Object Detection in the Deep Learning Era: An In-Depth Survey
• Boundary-Aware Segmentation Network for Mobile and Web Applications
• Co-salient Object Detection Based on Deep Saliency Networks and Seed Propagation over an Integrated Graph

Cited by in corpus (20)

• Concealed Object Detection
• RGB-D Salient Object Detection: A Survey
• Advances in Deep Concealed Scene Understanding
• Salient Objects in Clutter
• CycleSegNet: Object Co-segmentation with Cycle Refinement and Region Correspondence
• Local Label Point Correction for Edge Detection of Overlapping Cervical Cells
• Rethinking Object Saliency Ranking: A Novel Whole-flow Processing Paradigm
• Towards Stable Co-saliency Detection and Object Co-segmentation
• Mutual Graph Learning for Camouflaged Object Detection
• CoSformer: Detecting Co-Salient Object with Transformers
• Co-Salient Object Detection with Semantic-Level Consensus Extraction and Dispersion
• Can You Spot the Chameleon? Adversarially Camouflaging Images from Co-Salient Object Detection
• ASOD60K: An Audio-Induced Salient Object Detection Dataset for Panoramic Videos
• Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection
• Group Collaborative Learning for Co-Salient Object Detection
• Perception-and-Regulation Network for Salient Object Detection
• Complementary Segmentation of Primary Video Objects with Reversible Flows
• One-Shot Object Affordance Detection in the Wild
• Summarize and Search: Learning Consensus-aware Dynamic Convolution for Co-Saliency Detection
• Free Lunch for Co-Saliency Detection: Context Adjustment