Knowledge-Embedded Routing Network for Scene Graph Generation
arXiv:1903.03326
Abstract
To understand a scene in depth not only involves locating/recognizing individual objects, but also requires to infer the relationships and interactions among them. However, since the distribution of real-world relationships is seriously unbalanced, existing methods perform quite poorly for the less frequent relationships. In this work, we find that the statistical correlations between object pairs and their relationships can effectively regularize semantic space and make prediction less ambiguous, and thus well address the unbalanced distribution issue. To achieve this, we incorporate these statistical correlations into deep neural networks to facilitate scene graph generation by developing a Knowledge-Embedded Routing Network. More specifically, we show that the statistical correlations between objects appearing in images and their relationships, can be explicitly represented by a structured knowledge graph, and a routing mechanism is learned to propagate messages through the graph to explore their interactions. Extensive experiments on the large-scale Visual Genome dataset demonstrate the superiority of the proposed method over current state-of-the-art competitors.
Accepted by CVPR 2019
References in corpus (2)
Cited by in corpus (10)
- Knowledge Graph Transfer Network for Few-Shot Recognition
- Tackling the Challenges in Scene Graph Generation with Local-to-Global Interactions
- Learning by Abstraction: The Neural State Machine
- Meta Module Network for Compositional Visual Reasoning
- Dual ResGCN for Balanced Scene GraphGeneration
- Unbiased Scene Graph Generation via Rich and Fair Semantic Extraction
- CogTree: Cognition Tree Loss for Unbiased Scene Graph Generation
- Learning from the Scene and Borrowing from the Rich: Tackling the Long Tail in Scene Graph Generation
- Is Object Detection Necessary for Human-Object Interaction Recognition?
- Leveraging Auxiliary Text for Deep Recognition of Unseen Visual Relationships