VideoClick: Video Object Segmentation with a Single Click
arXiv:2101.06545
Abstract
Annotating videos with object segmentation masks typically involves a two stage procedure of drawing polygons per object instance for all the frames and then linking them through time. While simple, this is a very tedious, time consuming and expensive process, making the creation of accurate annotations at scale only possible for well-funded labs. What if we were able to segment an object in the full video with only a single click? This will enable video segmentation at scale with a very low budget opening the door to many applications. Towards this goal, in this paper we propose a bottom up approach where given a single click for each object in a video, we obtain the segmentation masks of these objects in the full video. In particular, we construct a correlation volume that assigns each pixel in a target frame to either one of the objects in the reference frame or the background. We then refine this correlation volume via a recurrent attention module and decode the final segmentation. To evaluate the performance, we label the popular and challenging Cityscapes dataset with video object segmentations. Results on this new CityscapesVideo dataset show that our approach outperforms all the baselines in this challenging setting.
References in corpus (11)
- Deep Residual Learning for Image Recognition
- Associative Embedding: End-to-End Learning for Joint Detection and Grouping
- Training Deep Nets with Sublinear Memory Cost
- Semantic Instance Segmentation with a Discriminative Loss Function
- YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark
- RANet: Ranking Attention Network for Fast Video Object Segmentation
- Semantic Instance Segmentation via Deep Metric Learning
- RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free
- Instance-aware Semantic Segmentation via Multi-task Network Cascades
- Learning a Spatio-Temporal Embedding for Video Instance Segmentation
- End-to-End Deep Structured Models for Drawing Crosswalks