Learning to discover and localize visual objects with open vocabulary
arXiv:1811.10080
Abstract
To alleviate the cost of obtaining accurate bounding boxes for training today's state-of-the-art object detection models, recent weakly supervised detection work has proposed techniques to learn from image-level labels. However, requiring discrete image-level labels is both restrictive and suboptimal. Real-world "supervision" usually consists of more unstructured text, such as captions. In this work we learn association maps between images and captions. We then use a novel objectness criterion to rank the resulting candidate boxes, such that high-ranking boxes have strong gradients along all edges. Thus, we can detect objects beyond a fixed object category vocabulary, if those objects are frequent and distinctive enough. We show that our objectness criterion improves the proposed bounding boxes in relation to prior weakly supervised detection methods. Further, we show encouraging results on object detection from image-level captions only.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Deep Learning for Generic Object Detection: A Survey
- Learning Deep Features for Discriminative Localization
- Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
- segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection
- Segmentation from Natural Language Expressions
- Learning to Separate Object Sounds by Watching Unlabeled Video
- A Hierarchical Approach for Generating Descriptive Image Paragraphs
- Discover and Learn New Objects from Documentaries