SegFlow: Joint Learning for Video Object Segmentation and Optical Flow

Abstract

This paper proposes an end-to-end trainable network, SegFlow, for simultaneously predicting pixel-wise object segmentation and optical flow in videos. The proposed SegFlow has two branches where useful information of object segmentation and optical flow is propagated bidirectionally in a unified framework. The segmentation branch is based on a fully convolutional network, which has been proved effective in image segmentation task, and the optical flow branch takes advantage of the FlowNet model. The unified framework is trained iteratively offline to learn a generic notion, and fine-tuned online for specific objects. Extensive experiments on both the video object segmentation and optical flow datasets demonstrate that introducing optical flow improves the performance of segmentation and vice versa, against the state-of-the-art algorithms.

Accepted in ICCV'17. Code is available at https://sites.google.com/site/yihsuantsai/research/iccv17-segflow

References in corpus (4)

• Deep Residual Learning for Image Recognition
• Fully Convolutional Networks for Semantic Segmentation
• DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
• Learning Video Object Segmentation from Static Images

Cited by in corpus (5)

• Learning Video Object Segmentation from Unlabeled Videos
• RadarMOSEVE: A Spatial-Temporal Transformer Network for Radar-Only Moving Object Segmentation and Ego-Velocity Estimation
• Joint Segmentation and Landmark Localization of Fetal Femur in Ultrasound Volumes
• Joint Learning of Instance and Semantic Segmentation for Robotic Pick-and-Place with Heavy Occlusions in Clutter
• RPM-Net: Robust Pixel-Level Matching Networks for Self-Supervised Video Object Segmentation