Dynamic Video Segmentation Network
arXiv:1804.00931
Abstract
In this paper, we present a detailed design of dynamic video segmentation network (DVSNet) for fast and efficient semantic video segmentation. DVSNet consists of two convolutional neural networks: a segmentation network and a flow network. The former generates highly accurate semantic segmentations, but is deeper and slower. The latter is much faster than the former, but its output requires further processing to generate less accurate semantic segmentations. We explore the use of a decision network to adaptively assign different frame regions to different networks based on a metric called expected confidence score. Frame regions with a higher expected confidence score traverse the flow network. Frame regions with a lower expected confidence score have to pass through the segmentation network. We have extensively performed experiments on various configurations of DVSNet, and investigated a number of variants for the proposed decision network. The experimental results show that our DVSNet is able to achieve up to 70.4% mIoU at 19.8 fps on the Cityscape dataset. A high speed version of DVSNet is able to deliver an fps of 30.4 with 63.2% mIoU on the same dataset. DVSNet is also able to reduce up to 95% of the computational workloads.
arXiv commentCVPR 2018
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Deep Residual Learning for Image Recognition
- Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
- Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
- Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- Multi-Scale Context Aggregation by Dilated Convolutions
- Going Deeper with Convolutions
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Fully Convolutional Networks for Semantic Segmentation
- Pyramid Scene Parsing Network
- Attention to Scale: Scale-aware Semantic Image Segmentation
- ICNet for Real-Time Semantic Segmentation on High-Resolution Images
- Optical Flow Estimation using a Spatial Pyramid Network