Learning to Segment Instances in Videos with Spatial Propagation Network
arXiv:1709.04609
Abstract
We propose a deep learning-based framework for instance-level object segmentation. Our method mainly consists of three steps. First, We train a generic model based on ResNet-101 for foreground/background segmentations. Second, based on this generic model, we fine-tune it to learn instance-level models and segment individual objects by using augmented object annotations in first frames of test videos. To distinguish different instances in the same video, we compute a pixel-level score map for each object from these instance-level models. Each score map indicates the objectness likelihood and is only computed within the foreground mask obtained in the first step. To further refine this per frame score map, we learn a spatial propagation network. This network aims to learn how to propagate a coarse segmentation mask spatially based on the pairwise similarities in each frame. In addition, we apply a filter on the refined score map that aims to recognize the best connected region using spatial and temporal consistencies in the video. Finally, we decide the instance-level object segmentation in each video by comparing score maps of different instances.
CVPR 2017 Workshop on DAVIS Challenge. Code is available at http://github.com/JingchunCheng/Seg-with-SPN
References in corpus (6)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Deep Residual Learning for Image Recognition
- Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
- Fully Convolutional Networks for Semantic Segmentation
- Instance-aware Semantic Segmentation via Multi-task Network Cascades
- Learning Video Object Segmentation from Static Images
Cited by in corpus (6)
- Video Object Segmentation and Tracking: A Survey
- Lucid Data Dreaming for Video Object Segmentation
- DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation
- Contextual Guided Segmentation Framework for Semi-supervised Video Instance Segmentation
- Fast and Accurate Online Video Object Segmentation via Tracking Parts
- Semantic Instance Meets Salient Object: Study on Video Semantic Salient Instance Segmentation