Efficient Coarse-to-Fine Non-Local Module for the Detection of Small Objects

Abstract

An image is not just a collection of objects, but rather a graph where each object is related to other objects through spatial and semantic relations. Using relational reasoning modules, such as the non-local module \cite{wang2017non}, can therefore improve object detection. Current schemes apply such dedicated modules either to a specific layer of the bottom-up stream, or between already-detected objects. We show that the relational process can be better modeled in a coarse-to-fine manner and present a novel framework, applying a non-local module sequentially to increasing resolution feature maps along the top-down stream. In this way, information can naturally passed from larger objects to smaller related ones. Applying the module to fine feature maps further allows the information to pass between the small objects themselves, exploiting repetitions of instances of the same class. In practice, due to the expensive memory utilization of the non-local module, it is infeasible to apply the module as currently used to high-resolution feature maps. We redesigned the non local module, improved it in terms of memory and number of operations, allowing it to be placed anywhere along the network. We further incorporated relative spatial information into the module, in a manner that can be incorporated into our efficient implementation. We show the effectiveness of our scheme by improving the results of detecting small objects on COCO by 1-2 AP points over Faster and Mask RCNN and by 1 AP over using non-local module on the bottom-up stream.

References in corpus (22)

• YOLOv3: An Incremental Improvement
• Deep Residual Learning for Image Recognition
• Geometric deep learning: going beyond Euclidean data
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Self-Attention Generative Adversarial Networks
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• DSSD : Deconvolutional Single Shot Detector
• Interaction Networks for Learning about Objects, Relations and Physics
• OCNet: Object Context Network for Scene Parsing
• Beyond Skip Connections: Top-Down Modulation for Object Detection
• Non-Local Recurrent Network for Image Restoration
• Non-local Neural Networks
• Compact Generalized Non-local Network
• Receptive Field Block Net for Accurate and Fast Object Detection
• DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection
• Discovering objects and their relations from entangled scene representations
• Relation Networks for Object Detection
• Iterative Visual Reasoning Beyond Convolutions
• Iterative Instance Segmentation
• Scene Parsing with Global Context Embedding
• Object-Level Context Modeling For Scene Classification with Context-CNN

Cited by in corpus (5)

• Hard-Attention for Scalable Image Classification
• Neural Architecture Search for Lightweight Non-Local Networks
• Multiple receptive fields and small-object-focusing weakly-supervised segmentation network for fast object detection
• Efficient Classification of Very Large Images with Tiny Objects
• HCNet: Hierarchical Context Network for Semantic Segmentation