IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things
arXiv:1906.06597
Abstract
In this work, we present a new operator, called Instance Mask Projection (IMP), which projects a predicted Instance Segmentation as a new feature for semantic segmentation. It also supports back propagation so is trainable end-to-end. Our experiments show the effectiveness of IMP on both Clothing Parsing (with complex layering, large deformations, and non-convex objects), and on Street Scene Segmentation (with many overlapping instances and small objects). On the Varied Clothing Parsing dataset (VCP), we show instance mask projection can improve 3 points on mIOU from a state-of-the-art Panoptic FPN segmentation approach. On the ModaNet clothing parsing dataset, we show a dramatic improvement of 20.4% absolutely compared to existing baseline semantic segmentation results. In addition, the instance mask projection operator works well on other (non-clothing) datasets, providing an improvement of 3 points in mIOU on Thing classes of Cityscapes, a self-driving dataset, on top of a state-of-the-art approach.
References in corpus (10)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- R-FCN: Object Detection via Region-based Fully Convolutional Networks
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Conditional Random Fields as Recurrent Neural Networks
- DSSD : Deconvolutional Single Shot Detector
- Fully Convolutional Networks for Semantic Segmentation
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
- Panoptic Feature Pyramid Networks
- UPSNet: A Unified Panoptic Segmentation Network