Object Relation Detection Based on One-shot Learning
arXiv:1807.05857
Abstract
Detecting the relations among objects, such as "cat on sofa" and "person ride horse", is a crucial task in image understanding, and beneficial to bridging the semantic gap between images and natural language. Despite the remarkable progress of deep learning in detection and recognition of individual objects, it is still a challenging task to localize and recognize the relations between objects due to the complex combinatorial nature of various kinds of object relations. Inspired by the recent advances in one-shot learning, we propose a simple yet effective Semantics Induced Learner (SIL) model for solving this challenging task. Learning in one-shot manner can enable a detection model to adapt to a huge number of object relations with diverse appearance effectively and robustly. In addition, the SIL combines bottom-up and top-down attention mech- anisms, therefore enabling attention at the level of vision and semantics favorably. Within our proposed model, the bottom-up mechanism, which is based on Faster R-CNN, proposes objects regions, and the top-down mechanism selects and integrates visual features according to semantic information. Experiments demonstrate the effectiveness of our framework over other state-of-the-art methods on two large-scale data sets for object relation detection.
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- VQA: Visual Question Answering
- Learning feed-forward one-shot learners
- Visual Relationship Detection with Language Priors
- Detecting Visual Relationships with Deep Relational Networks
- Visual Translation Embedding Network for Visual Relation Detection
- From Images to Sentences through Scene Description Graphs using Commonsense Reasoning and Knowledge