Visual Relationship Detection using Scene Graphs: A Survey
arXiv:2005.08045
Abstract
Understanding a scene by decoding the visual relationships depicted in an image has been a long studied problem. While the recent advances in deep learning and the usage of deep neural networks have achieved near human accuracy on many tasks, there still exists a pretty big gap between human and machine level performance when it comes to various visual relationship detection tasks. Developing on earlier tasks like object recognition, segmentation and captioning which focused on a relatively coarser image understanding, newer tasks have been introduced recently to deal with a finer level of image understanding. A Scene Graph is one such technique to better represent a scene and the various relationships present in it. With its wide number of applications in various tasks like Visual Question Answering, Semantic Image Retrieval, Image Generation, among many others, it has proved to be a useful tool for deeper and better visual relationship understanding. In this paper, we present a detailed survey on the various techniques for scene graph generation, their efficacy to represent visual relationships and how it has been used to solve various downstream tasks. We also attempt to analyze the various future directions in which the field might advance in the future. Being one of the first papers to give a detailed survey on this topic, we also hope to give a succinct introduction to scene graphs, and guide practitioners while developing approaches for their applications.
References in corpus (22)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Semi-Supervised Classification with Graph Convolutional Networks
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- YOLOv3: An Incremental Improvement
- R-FCN: Object Detection via Region-based Fully Convolutional Networks
- Spectral Networks and Locally Connected Networks on Graphs
- YFCC100M: The New Data in Multimedia Research
- VQA: Visual Question Answering
- Density estimation using Real NVP
- Object Detectors Emerge in Deep Scene CNNs
- Weakly Supervised Object Localization with Multi-fold Multiple Instance Learning
- Gated Graph Sequence Neural Networks
- A Short Note on the Kinetics-700-2020 Human Action Dataset
- Light-Head R-CNN: In Defense of Two-Stage Object Detector
- GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
- Decoupling Representation and Classifier for Long-Tailed Recognition
- Learning Deep Features for Discriminative Localization
- Unbiased Scene Graph Generation from Biased Training
- Generating Natural Language Explanations for Visual Question Answering using Scene Graphs and Visual Attention
- Action Genome: Actions as Composition of Spatio-temporal Scene Graphs