MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features

Abstract

In this work, we tackle the problem of instance segmentation, the task of simultaneously solving object detection and semantic segmentation. Towards this goal, we present a model, called MaskLab, which produces three outputs: box detection, semantic segmentation, and direction prediction. Building on top of the Faster-RCNN object detector, the predicted boxes provide accurate localization of object instances. Within each region of interest, MaskLab performs foreground/background segmentation by combining semantic and direction prediction. Semantic segmentation assists the model in distinguishing between objects of different semantic classes including background, while the direction prediction, estimating each pixel's direction towards its corresponding center, allows separating instances of the same semantic class. Moreover, we explore the effect of incorporating recent successful methods from both segmentation and detection (i.e. atrous convolution and hypercolumn). Our proposed model is evaluated on the COCO instance segmentation benchmark and shows comparable performance with other state-of-art models.

10 pages including reference

References in corpus (28)

• Distilling the Knowledge in a Neural Network
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Deep Residual Learning for Image Recognition
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Semantic Instance Segmentation with a Discriminative Loss Function
• Deformable Convolutional Networks
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Xception: Deep Learning with Depthwise Separable Convolutions
• Scalable, High-Quality Object Detection
• Beyond Skip Connections: Top-Down Modulation for Object Detection
• Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• Semantic Instance Segmentation via Deep Metric Learning
• Semantic Image Segmentation via Deep Parsing Network
• Bridging Category-level and Instance-level Semantic Image Segmentation
• Proposal-free Network for Instance-level Object Segmentation
• Instance-sensitive Fully Convolutional Networks
• Fast Image Scanning with Deep Max-Pooling Convolutional Neural Networks
• segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection
• Understanding Convolution for Semantic Segmentation
• Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling
• Monocular Object Instance Segmentation and Depth Ordering with CNNs
• Object Detection Free Instance Segmentation With Labeling Transformations

Cited by in corpus (8)

• A Review of Object Detection Models based on Convolutional Neural Network
• Learning to Fuse Things and Stuff
• ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors
• Actor-Action Semantic Segmentation with Region Masks
• The NEOLIX Open Dataset for Autonomous Driving
• MaskPlus: Improving Mask Generation for Instance Segmentation
• Affinity Derivation and Graph Merge for Instance Segmentation
• Accurate Nuclear Segmentation with Center Vector Encoding