Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generation
arXiv:1806.11538
Abstract
Generating scene graph to describe all the relations inside an image gains increasing interests these years. However, most of the previous methods use complicated structures with slow inference speed or rely on the external data, which limits the usage of the model in real-life scenarios. To improve the efficiency of scene graph generation, we propose a subgraph-based connection graph to concisely represent the scene graph during the inference. A bottom-up clustering method is first used to factorize the entire scene graph into subgraphs, where each subgraph contains several objects and a subset of their relationships. By replacing the numerous relationship representations of the scene graph with fewer subgraph and object features, the computation in the intermediate stage is significantly reduced. In addition, spatial information is maintained by the subgraph features, which is leveraged by our proposed Spatial-weighted Message Passing~(SMP) structure and Spatial-sensitive Relation Inference~(SRI) module to facilitate the relationship recognition. On the recent Visual Relationship Detection and Visual Genome datasets, our method outperforms the state-of-the-art method in both accuracy and speed.
arXiv commentECCV 2018
Cited by in corpus (11)
- DeepGCNs: Making GCNs Go as Deep as CNNs
- PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph
- An Empirical Study on Leveraging Scene Graphs for Visual Question Answering
- Tackling the Challenges in Scene Graph Generation with Local-to-Global Interactions
- Attentive Relational Networks for Mapping Images to Scene Graphs
- Unbiased Scene Graph Generation via Rich and Fair Semantic Extraction
- ORD: Object Relationship Discovery for Visual Dialogue Generation
- Combining Optimal Path Search With Task-Dependent Learning in a Neural Network
- Relating Blindsight and AI: A Review
- Assisting Scene Graph Generation with Self-Supervision
- Target-Tailored Source-Transformation for Scene Graph Generation