Efficient Video Object Segmentation via Network Modulation

Abstract

Video object segmentation targets at segmenting a specific object throughout a video sequence, given only an annotated first frame. Recent deep learning based approaches find it effective by fine-tuning a general-purpose segmentation model on the annotated frame using hundreds of iterations of gradient descent. Despite the high accuracy these methods achieve, the fine-tuning process is inefficient and fail to meet the requirements of real world applications. We propose a novel approach that uses a single forward pass to adapt the segmentation model to the appearance of a specific object. Specifically, a second meta neural network named modulator is learned to manipulate the intermediate layers of the segmentation network given limited visual and spatial information of the target object. The experiments show that our approach is 70times faster than fine-tuning approaches while achieving similar accuracy.

arXiv commentSubmitted to CVPR 2018

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Matching Networks for One Shot Learning
• A Learned Representation For Artistic Style
• Deformable Convolutional Networks
• Modulating early visual processing by language
• FiLM: Visual Reasoning with a General Conditioning Layer
• Learning to Learn without Gradient Descent by Gradient Descent
• Learning Video Object Segmentation with Visual Memory
• Video Propagation Networks

Cited by in corpus (11)

• Slimmable Neural Networks
• Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation
• Space-Time Correspondence as a Contrastive Random Walk
• Do Different Tracking Tasks Require Different Appearance Models?
• Learning Video Object Segmentation from Unlabeled Videos
• Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning
• Semantics Disentangling for Text-to-Image Generation
• Learning Correspondence from the Cycle-Consistency of Time
• Crossover Learning for Fast Online Video Instance Segmentation
• Discriminative Online Learning for Fast Video Object Segmentation
• RPM-Net: Robust Pixel-Level Matching Networks for Self-Supervised Video Object Segmentation