Transformation-Based Models of Video Sequences
arXiv:1701.08435
Abstract
In this work we propose a simple unsupervised approach for next frame prediction in video. Instead of directly predicting the pixels in a frame given past frames, we predict the transformations needed for generating the next frame in a sequence, given the transformations of the past frames. This leads to sharper results, while using a smaller prediction model. In order to enable a fair comparison between different video frame prediction models, we also propose a new evaluation protocol. We use generated frames as input to a classifier trained with ground truth sequences. This criterion guarantees that models scoring high are those producing sequences which preserve discriminative features, as opposed to merely penalizing any deviation, plausible or not, from the ground truth. Our proposed approach compares favourably against more sophisticated ones on the UCF-101 data set, while also being more efficient in terms of the number of parameters and computational cost.
References in corpus (2)
Cited by in corpus (22)
- A Review on Deep Learning Techniques for Video Prediction
- Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
- Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge
- Stochastic Adversarial Video Prediction
- Learning to Decompose and Disentangle Representations for Video Prediction
- Future Frame Prediction for Anomaly Detection -- A New Baseline
- Predicting Deeper into the Future of Semantic Segmentation
- FitVid: Overfitting in Pixel-Level Video Prediction
- Video Imagination from a Single Image with Transformation Generation
- From Here to There: Video Inbetweening Using Direct 3D Convolutions
- Transformation-based Adversarial Video Prediction on Large-Scale Data
- Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction
- Effect of Architectures and Training Methods on the Performance of Learned Video Frame Prediction
- Exploring Spatial-Temporal Multi-Frequency Analysis for High-Fidelity and Temporal-Consistency Video Prediction
- Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction
- Frequency Domain Transformer Networks for Video Prediction
- Deep Learned Frame Prediction for Video Compression
- DeepLandscape: Adversarial Modeling of Landscape Video
- Efficient training for future video generation based on hierarchical disentangled representation of latent variables
- Brain-Inspired Inference on Missing Video Sequence
- SDCNet: Video Prediction Using Spatially-Displaced Convolution
- VP-GO: a "light" action-conditioned visual prediction model