Visual and Semantic Knowledge Transfer for Large Scale Semi-supervised Object Detection

Abstract

Deep CNN-based object detection systems have achieved remarkable success on several large-scale object detection benchmarks. However, training such detectors requires a large number of labeled bounding boxes, which are more difficult to obtain than image-level annotations. Previous work addresses this issue by transforming image-level classifiers into object detectors. This is done by modeling the differences between the two on categories with both image-level and bounding box annotations, and transferring this information to convert classifiers to detectors for categories without bounding box annotations. We improve this previous work by incorporating knowledge about object similarities from visual and semantic domains during the transfer process. The intuition behind our proposed method is that visually and semantically similar categories should exhibit more common transferable properties than dissimilar categories, e.g. a better detector would result by transforming the differences between a dog classifier and a dog detector onto the cat class, than would by transforming from the violin class. Experimental results on the challenging ILSVRC2013 detection dataset demonstrate that each of our proposed object similarity based knowledge transfer methods outperforms the baseline methods. We found strong evidence that visual similarity and semantic relatedness are complementary for the task, and when combined notably improve detection, achieving state-of-the-art detection performance in a semi-supervised setting.

TPAMI. correct some typos

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Object Detectors Emerge in Deep Scene CNNs
• AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes

Cited by in corpus (11)

• Discriminative and Geometry Aware Unsupervised Domain Adaptation
• Attention Regularized Laplace Graph for Domain Adaptation
• Boosting Weakly Supervised Object Detection with Progressive Knowledge Transfer
• TUNA-Net: Task-oriented UNsupervised Adversarial Network for Disease Recognition in Cross-Domain Chest X-rays
• Scaling Object Detection by Transferring Classification Weights
• One-Click Annotation with Guided Hierarchical Object Detection
• MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection
• Discriminative Noise Robust Sparse Orthogonal Label Regression-based Domain Adaptation
• Decoupled Box Proposal and Featurization with Ultrafine-Grained Semantic Labels Improve Image Captioning and Visual Question Answering
• CS-R-FCN: Cross-supervised Learning for Large-Scale Object Detection
• CHEER: Rich Model Helps Poor Model via Knowledge Infusion