Semi-convolutional Operators for Instance Segmentation

Abstract

Object detection and instance segmentation are dominated by region-based methods such as Mask RCNN. However, there is a growing interest in reducing these problems to pixel labeling tasks, as the latter could be more efficient, could be integrated seamlessly in image-to-image network architectures as used in many other tasks, and could be more accurate for objects that are not well approximated by bounding boxes. In this paper we show theoretically and empirically that constructing dense pixel embeddings that can separate object instances cannot be easily achieved using convolutional operators. At the same time, we show that simple modifications, which we call semi-convolutional, have a much better chance of succeeding at this task. We use the latter to show a connection to Hough voting as well as to a variant of the bilateral kernel that is spatially steered by a convolutional network. We demonstrate that these operators can also be used to improve approaches such as Mask RCNN, demonstrating better segmentation of complex biological shapes and PASCAL VOC categories than achievable by Mask RCNN alone.

arXiv commentAccepted as a conference paper at ECCV 2018

References in corpus (14)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Learning to Segment Object Candidates
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Semantic Instance Segmentation with a Discriminative Loss Function
• Semantic Instance Segmentation via Deep Metric Learning
• Simultaneous Detection and Segmentation
• Proposal-free Network for Instance-level Object Segmentation
• Instance-sensitive Fully Convolutional Networks
• Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling
• Monocular Object Instance Segmentation and Depth Ordering with CNNs
• Learning Dense Convolutional Embeddings for Semantic Segmentation
• Parsing Images of Overlapping Organisms with Deep Singling-Out Networks

Cited by in corpus (4)

• Learning Gaussian Instance Segmentation in Point Clouds
• TextMountain: Accurate Scene Text Detection via Instance Segmentation
• Can we cover navigational perception needs of the visually impaired by panoptic segmentation?
• supervised adptive threshold network for instance segmentation