Semantics-Guided Contrastive Network for Zero-Shot Object detection

Abstract

Zero-shot object detection (ZSD), the task that extends conventional detection models to detecting objects from unseen categories, has emerged as a new challenge in computer vision. Most existing approaches tackle the ZSD task with a strict mapping-transfer strategy, which may lead to suboptimal ZSD results: 1) the learning process of those models ignores the available unseen class information, and thus can be easily biased towards the seen categories; 2) the original visual feature space is not well-structured and lack of discriminative information. To address these issues, we develop a novel Semantics-Guided Contrastive Network for ZSD, named ContrastZSD, a detection framework that first brings contrastive learning mechanism into the realm of zero-shot detection. Particularly, ContrastZSD incorporates two semantics-guided contrastive learning subnets that contrast between region-category and region-region pairs respectively. The pairwise contrastive tasks take advantage of additional supervision signals derived from both ground truth label and pre-defined class similarity distribution. Under the guidance of those explicit semantic supervision, the model can learn more knowledge about unseen categories to avoid the bias problem to seen concepts, while optimizing the data structure of visual features to be more discriminative for better visual-semantic alignment. Extensive experiments are conducted on two popular benchmarks for ZSD, i.e., PASCAL VOC and MS COCO. Results show that our method outperforms the previous state-of-the-art on both ZSD and generalized ZSD tasks.

15 pages, Accepted

References in corpus (10)

• A Simple Framework for Contrastive Learning of Visual Representations
• Improved Baselines with Momentum Contrastive Learning
• Self-supervised Learning: Generative or Contrastive
• Zero-Shot Learning Through Cross-Modal Transfer
• What Makes for Good Views for Contrastive Learning?
• Transductive Multi-view Zero-Shot Learning
• A Unified Perspective on Multi-Domain and Multi-Task Learning
• Spatiotemporal Contrastive Video Representation Learning
• Background Learnable Cascade for Zero-Shot Object Detection
• Attribute-Modulated Generative Meta Learning for Zero-Shot Classification

Cited by in corpus (4)

• Fine-grained Visual-Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection
• Synthesizing Knowledge-enhanced Features for Real-world Zero-shot Food Detection
• SeeDS: Semantic Separable Diffusion Synthesizer for Zero-shot Food Detection
• Generalized Semantic Contrastive Learning via Embedding Side Information for Few-Shot Object Detection