Obj-GloVe: Scene-Based Contextual Object Embedding
arXiv:1907.01478
Abstract
Recently, with the prevalence of large-scale image dataset, the co-occurrence information among classes becomes rich, calling for a new way to exploit it to facilitate inference. In this paper, we propose Obj-GloVe, a generic scene-based contextual embedding for common visual objects, where we adopt the word embedding method GloVe to exploit the co-occurrence between entities. We train the embedding on pre-processed Open Images V4 dataset and provide extensive visualization and analysis by dimensionality reduction and projecting the vectors along a specific semantic axis, and showcasing the nearest neighbors of the most common objects. Furthermore, we reveal the potential applications of Obj-GloVe on object detection and text-to-image synthesis, then verify its effectiveness on these two applications respectively.
arXiv comment14 pages; not the final version
References in corpus (6)
- Efficient Estimation of Word Representations in Vector Space
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Distributed Representations of Words and Phrases and their Compositionality
- Generative Adversarial Text to Image Synthesis
- The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
- Pair-Linking for Collective Entity Disambiguation: Two Could Be Better Than All