Temporally Distributed Networks for Fast Video Semantic Segmentation
arXiv:2004.01800
Abstract
We present TDNet, a temporally distributed network designed for fast and accurate video semantic segmentation. We observe that features extracted from a certain high-level layer of a deep CNN can be approximated by composing features extracted from several shallower sub-networks. Leveraging the inherent temporal continuity in videos, we distribute these sub-networks over sequential frames. Therefore, at each time step, we only need to perform a lightweight computation to extract a sub-features group from a single sub-network. The full features used for segmentation are then recomposed by application of a novel attention propagation module that compensates for geometry deformation between frames. A grouped knowledge distillation loss is also introduced to further improve the representation power at both full and sub-feature levels. Experiments on Cityscapes, CamVid, and NYUD-v2 demonstrate that our method achieves state-of-the-art accuracy with significantly faster speed and lower latency.
[CVPR2020] Project: https://github.com/feinanshan/TDNet
References in corpus (9)
- Distilling the Knowledge in a Neural Network
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
- Context Encoding for Semantic Segmentation
- DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation
- Normalized Cut Loss for Weakly-supervised CNN Segmentation
- Knowledge Adaptation for Efficient Semantic Segmentation