Scenes and Surroundings: Scene Graph Generation using Relation Transformer
arXiv:2107.05448
Abstract
Identifying objects in an image and their mutual relationships as a scene graph leads to a deep understanding of image content. Despite the recent advancement in deep learning, the detection and labeling of visual object relationships remain a challenging task. This work proposes a novel local-context aware architecture named relation transformer, which exploits complex global objects to object and object to edge (relation) interactions. Our hierarchical multi-head attention-based approach efficiently captures contextual dependencies between objects and predicts their relationships. In comparison to state-of-the-art approaches, we have achieved an overall mean \textbf{4.85\%} improvement and a new benchmark across all the scene graph generation tasks on the Visual Genome dataset.
arXiv commentarXiv admin note: text overlap with arXiv:2004.06193
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Empirical Evaluation of Rectified Activations in Convolutional Network
- Scene Graph Generation by Iterative Message Passing
- Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships
- Generating Natural Language Explanations for Visual Question Answering using Scene Graphs and Visual Attention
- Large-Scale Visual Relationship Understanding
- Scene Graph Reasoning for Visual Question Answering
- Relation Transformer Network