Interactive Video Object Segmentation in the Wild

Abstract

In this paper we present our system for human-in-the-loop video object segmentation. The backbone of our system is a method for one-shot video object segmentation. While fast, this method requires an accurate pixel-level segmentation of one (or several) frames as input. As manually annotating such a segmentation is impractical, we propose a deep interactive image segmentation method, that can accurately segment objects with only a handful of clicks. On the GrabCut dataset, our method obtains 90% IOU with just 3.8 clicks on average, setting the new state of the art. Furthermore, as our method iteratively refines an initial segmentation, it can effectively correct frames where the video object segmentation fails, thus allowing users to quickly obtain high quality results even on challenging sequences. Finally, we investigate usage patterns and give insights in how many steps users take to annotate frames, what kind of corrections they provide, etc., thus giving important insights for further improving interactive video segmentation.

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Deep Interactive Object Selection
• Lucid Data Dreaming for Video Object Segmentation
• Deep Extreme Cut: From Extreme Points to Object Segmentation

Cited by in corpus (12)

• A Survey of Human-in-the-loop for Machine Learning
• Video Object Segmentation and Tracking: A Survey
• Getting to 99% Accuracy in Interactive Segmentation
• Chan-Vese Reformulation for Selective Image Segmentation
• Memory Aggregation Networks for Efficient Interactive Video Object Segmentation
• Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion
• Efficient Full Image Interactive Segmentation by Leveraging Within-image Appearance Similarity
• ScribbleBox: Interactive Annotation Framework for Video Object Segmentation
• Guided Interactive Video Object Segmentation Using Reliability-Based Attention Maps
• Localized Interactive Instance Segmentation
• Interactive Video Object Segmentation Using Global and Local Transfer Modules
• Scale-aware multi-level guidance for interactive instance segmentation