Mutual Suppression Network for Video Prediction using Disentangled Features
arXiv:1804.04810
Abstract
Video prediction has been considered a difficult problem because the video contains not only high-dimensional spatial information but also complex temporal information. Video prediction can be performed by finding features in recent frames, and using them to generate approximations to upcoming frames. We approach this problem by disentangling spatial and temporal features in videos. We introduce a mutual suppression network (MSnet) which are trained in an adversarial manner and then produces spatial features which are free of motion information, and motion features with no spatial information. MSnet then uses motion-guided connection within an encoder-decoder-based architecture to transform spatial features from a previous frame to the time of an upcoming frame. We show how MSnet can be used for video prediction using disentangled representations. We also carry out experiments to assess the effectiveness of our method to disentangle features. MSnet obtains better results than other recent video prediction methods even though it has simpler encoders.
arXiv commentBMVC 2019 (Oral)
References in corpus (10)
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Unsupervised Learning of Video Representations using LSTMs
- Spatiotemporal Residual Networks for Video Action Recognition
- Decomposing Motion and Content for Natural Video Sequence Prediction
- ConvNet Architecture Search for Spatiotemporal Feature Learning
- Video (language) modeling: a baseline for generative models of natural videos
- FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos
- An End-to-end 3D Convolutional Neural Network for Action Detection and Segmentation in Videos
- Spatio-Temporal Action Detection with Cascade Proposal and Location Anticipation