Video Propagation Networks

Abstract

We propose a technique that propagates information forward through video data. The method is conceptually simple and can be applied to tasks that require the propagation of structured information, such as semantic labels, based on video content. We propose a 'Video Propagation Network' that processes video frames in an adaptive manner. The model is applied online: it propagates information forward without the need to access future frames. In particular we combine two components, a temporal bilateral network for dense and video adaptive filtering, followed by a spatial network to refine features and increased flexibility. We present experiments on video object segmentation and semantic video segmentation and show increased performance comparing to the best previous task-specific methods, while having favorable runtime. Additionally we demonstrate our approach on an example regression task of color propagation in a grayscale video.

Appearing in Computer Vision and Pattern Recognition, 2017 (CVPR'17)

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Multi-Scale Context Aggregation by Dilated Convolutions
• Fully Convolutional Networks for Semantic Segmentation
• Fully Connected Deep Structured Networks
• Playing for Data: Ground Truth from Computer Games
• Learning Graphical Model Parameters with Approximate Marginal Inference
• Learning Video Object Segmentation from Static Images
• gSLICr: SLIC superpixels at over 250Hz
• Mean-Field Networks

Cited by in corpus (13)

• RANet: Ranking Attention Network for Fast Video Object Segmentation
• Fast Online Object Tracking and Segmentation: A Unifying Approach
• Efficient Video Object Segmentation via Network Modulation
• Lucid Data Dreaming for Video Object Segmentation
• Instance Embedding Transfer to Unsupervised Video Object Segmentation
• Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation
• Video Object Segmentation using Supervoxel-Based Gerrymandering
• Fully Automatic Video Colorization with Self-Regularization and Diversity
• See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks
• Fast and Accurate Online Video Object Segmentation via Tracking Parts
• Semantic Segmentation via Highly Fused Convolutional Network with Multiple Soft Cost Functions
• Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow
• CNN in MRF: Video Object Segmentation via Inference in A CNN-Based Higher-Order Spatio-Temporal MRF