Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships

Abstract

Context is important for accurate visual recognition. In this work we propose an object detection algorithm that not only considers object visual appearance, but also makes use of two kinds of context including scene contextual information and object relationships within a single image. Therefore, object detection is regarded as both a cognition problem and a reasoning problem when leveraging these structured information. Specifically, this paper formulates object detection as a problem of graph structure inference, where given an image the objects are treated as nodes in a graph and relationships between the objects are modeled as edges in such graph. To this end, we present a so-called Structure Inference Network (SIN), a detector that incorporates into a typical detection framework (e.g. Faster R-CNN) with a graphical model which aims to infer object state. Comprehensive experiments on PASCAL VOC and MS COCO datasets indicate that scene context and object relationships truly improve the performance of object detection with more desirable and reasonable outputs.

arXiv commentpublished in CVPR 2018

References in corpus (13)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Focal Loss for Dense Object Detection
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Feature Pyramid Networks for Object Detection
• YOLO9000: Better, Faster, Stronger
• HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection
• Scene Graph Generation by Iterative Message Passing
• DSOD: Learning Deeply Supervised Object Detectors from Scratch
• Structured Sequence Modeling with Graph Convolutional Recurrent Networks
• Exploring Person Context and Local Scene Context for Object Detection
• Structure Inference Machines: Recurrent Neural Networks for Analyzing Relations in Group Activity Recognition
• Learning Structured Inference Neural Networks with Label Relations

Cited by in corpus (6)

• Deep Learning for Generic Object Detection: A Survey
• A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets
• Scenes and Surroundings: Scene Graph Generation using Relation Transformer
• HR-RCNN: Hierarchical Relational Reasoning for Object Detection
• Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN
• GoG: Relation-aware Graph-over-Graph Network for Visual Dialog