SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation
arXiv:2007.14772
Abstract
Single-stage instance segmentation approaches have recently gained popularity due to their speed and simplicity, but are still lagging behind in accuracy, compared to two-stage methods. We propose a fast single-stage instance segmentation method, called SipMask, that preserves instance-specific spatial information by separating mask prediction of an instance to different sub-regions of a detected bounding-box. Our main contribution is a novel light-weight spatial preservation (SP) module that generates a separate set of spatial coefficients for each sub-region within a bounding-box, leading to improved mask predictions. It also enables accurate delineation of spatially adjacent instances. Further, we introduce a mask alignment weighting loss and a feature alignment scheme to better correlate mask prediction with object detection. On COCO test-dev, our SipMask outperforms the existing single-stage methods. Compared to the state-of-the-art single-stage TensorMask, SipMask obtains an absolute gain of 1.0% (mask AP), while providing a four-fold speedup. In terms of real-time capabilities, SipMask outperforms YOLACT with an absolute gain of 3.0% (mask AP) under similar settings, while operating at comparable speed on a Titan Xp. We also evaluate our SipMask for real-time video instance segmentation, achieving promising results on YouTube-VIS dataset. The source code is available at https://github.com/JialeCao001/SipMask.
ECCV2020; Code: https://github.com/JialeCao001/SipMask
References in corpus (7)
- Deep Residual Learning for Image Recognition
- R-FCN: Object Detection via Region-based Fully Convolutional Networks
- YOLACT++: Better Real-time Instance Segmentation
- Hybrid Task Cascade for Instance Segmentation
- RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free
- PolarMask: Single Shot Instance Segmentation with Polar Representation
- RDSNet: A New Deep Architecture for Reciprocal Object Detection and Instance Segmentation
Cited by in corpus (8)
- Video Instance Segmentation using Inter-Frame Communication Transformers
- Do Different Tracking Tasks Require Different Appearance Models?
- SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation
- Track to Detect and Segment: An Online Multi-Object Tracker
- Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentation
- BoundarySqueeze: Image Segmentation as Boundary Squeezing
- Automatic Generation of Dense Non-rigid Optical Flow
- MSN: Efficient Online Mask Selection Network for Video Instance Segmentation