Zero-Shot Detection

Abstract

As we move towards large-scale object detection, it is unrealistic to expect annotated training data, in the form of bounding box annotations around objects, for all object classes at sufficient scale, and so methods capable of unseen object detection are required. We propose a novel zero-shot method based on training an end-to-end model that fuses semantic attribute prediction with visual features to propose object bounding boxes for seen and unseen classes. While we utilize semantic features during training, our method is agnostic to semantic information for unseen classes at test-time. Our method retains the efficiency and effectiveness of YOLOv2 for objects seen during training, while improving its performance for novel and unseen objects. The ability of state-of-art detection methods to learn discriminative object features to reject background proposals also limits their performance for unseen objects. We posit that, to detect unseen objects, we must incorporate semantic information into the visual domain so that the learned visual features reflect this information and leads to improved recall rates for unseen objects. We test our method on PASCAL VOC and MS COCO dataset and observed significant improvements on the average precision of unseen classes.

arXiv commentPublished in IEEE Transactions on Circuits and Systems for Video Technology

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SSD: Single Shot MultiBox Detector
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distributed Representations of Words and Phrases and their Compositionality
• Focal Loss for Dense Object Detection
• Zero-Shot Learning Through Cross-Modal Transfer
• Label-Embedding for Image Classification
• Evaluation of Output Embeddings for Fine-Grained Image Classification
• Object Detectors Emerge in Deep Scene CNNs
• YOLO9000: Better, Faster, Stronger
• Recent Advances in Zero-shot Recognition
• Probabilistic Semantic Retrieval for Surveillance Videos with Activity Graphs
• Generalized Zero-Shot Recognition based on Visually Semantic Embedding
• Learning Classifiers for Domain Adaptation, Zero and Few-Shot Recognition Based on Learning Latent Semantic Parts

Cited by in corpus (20)

• Few-Example Object Detection with Model Communication
• One-Shot Instance Segmentation
• Small Sample Learning in Big Data Era
• Background Learnable Cascade for Zero-Shot Object Detection
• Polarity Loss for Zero-shot Object Detection
• Deep Semantic-Visual Alignment for Zero-Shot Remote Sensing Image Scene Classification
• Open-Vocabulary Object Detection Using Captions
• Few-shot Object Detection via Feature Reweighting
• Lesion Guided Explainable Few Weak-shot Medical Report Generation
• OMNIA Faster R-CNN: Detection in the wild through dataset merging and soft distillation
• Zero-Shot Semantic Segmentation
• Generalized Semantic Contrastive Learning via Embedding Side Information for Few-Shot Object Detection
• Dont Even Look Once: Synthesizing Features for Zero-Shot Detection
• Context-Aware Zero-Shot Recognition
• Zero-sample surface defect detection and classification based on semantic feedback neural network
• Weak Novel Categories without Tears: A Survey on Weak-Shot Learning
• ZSTAD: Zero-Shot Temporal Activity Detection
• Top-Related Meta-Learning Method for Few-Shot Object Detection
• Zero-Shot Instance Segmentation
• Incrementally Zero-Shot Detection by an Extreme Value Analyzer