Order Matters: Shuffling Sequence Generation for Video Prediction
arXiv:1907.08845
Abstract
Predicting future frames in natural video sequences is a new challenge that is receiving increasing attention in the computer vision community. However, existing models suffer from severe loss of temporal information when the predicted sequence is long. Compared to previous methods focusing on generating more realistic contents, this paper extensively studies the importance of sequential order information for video generation. A novel Shuffling sEquence gEneration network (SEE-Net) is proposed that can learn to discriminate unnatural sequential orders by shuffling the video frames and comparing them to the real video sequence. Systematic experiments on three datasets with both synthetic and real-world videos manifest the effectiveness of shuffling sequence generation for video prediction in our proposed model and demonstrate state-of-the-art performance by both qualitative and quantitative evaluations. The source code is available at https://github.com/andrewjywang/SEENet.
This manuscript has been accepted at BMVC 2019. See the project at https://github.com/andrewjywang/SEENet
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Unsupervised Learning of Video Representations using LSTMs
- Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
- Decomposing Motion and Content for Natural Video Sequence Prediction
- Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
- Deep multi-scale video prediction beyond mean square error
- Spatio-temporal video autoencoder with differentiable memory
- Stochastic Adversarial Video Prediction
- Stochastic Variational Video Prediction
- Robust Cross-View Gait Recognition with Evidence: A Discriminant Gait GAN (DiGGAN) Approach