Noisy-LSTM: Improving Temporal Awareness for Video Semantic Segmentation

Abstract

Semantic video segmentation is a key challenge for various applications. This paper presents a new model named Noisy-LSTM, which is trainable in an end-to-end manner, with convolutional LSTMs (ConvLSTMs) to leverage the temporal coherency in video frames. We also present a simple yet effective training strategy, which replaces a frame in video sequence with noises. This strategy spoils the temporal coherency in video frames during training and thus makes the temporal links in ConvLSTMs unreliable, which may consequently improve feature extraction from video frames, as well as serve as a regularizer to avoid overfitting, without requiring extra data annotation or computational costs. Experimental results demonstrate that the proposed model can achieve state-of-the-art performances in both the CityScapes and EndoVis2018 datasets.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Improving neural networks by preventing co-adaptation of feature detectors
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Self-training with Noisy Student improves ImageNet classification
• 2018 Robotic Scene Segmentation Challenge
• Improving Fast Segmentation With Teacher-student Learning
• Predicting Scene Parsing and Motion Dynamics in the Future
• Survey of Dropout Methods for Deep Neural Networks
• Future Semantic Segmentation with Convolutional LSTM