An Interpretable Model for Scene Graph Generation
arXiv:1811.09543
Abstract
We propose an efficient and interpretable scene graph generator. We consider three types of features: visual, spatial and semantic, and we use a late fusion strategy such that each feature's contribution can be explicitly investigated. We study the key factors about these features that have the most impact on the performance, and also visualize the learned visual features for relationships and investigate the efficacy of our model. We won the champion of the OpenImages Visual Relationship Detection Challenge on Kaggle, where we outperform the 2nd place by 5\% (20\% relatively). We believe an accurate scene graph generator is a fundamental stepping stone for higher-level vision-language tasks such as image captioning and visual QA, since it provides a semantic, structured comprehension of an image that is beyond pixels and objects.
arXiv commentarXiv admin note: substantial text overlap with arXiv:1811.00662
References in corpus (13)
- VQA: Visual Question Answering
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
- Pixels to Graphs by Associative Embedding
- Visual Relationship Detection with Language Priors
- Scene Graph Generation by Iterative Message Passing
- Detecting Visual Relationships with Deep Relational Networks
- Visual Translation Embedding Network for Visual Relation Detection
- Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation
- Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection
- PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCN
- Weakly-supervised learning of visual relations
- Referring Relationships
- Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition