Hierarchical Video Generation for Complex Data
arXiv:2106.02719
Abstract
Videos can often be created by first outlining a global description of the scene and then adding local details. Inspired by this we propose a hierarchical model for video generation which follows a coarse to fine approach. First our model generates a low resolution video, establishing the global scene structure, that is then refined by subsequent levels in the hierarchy. We train each level in our hierarchy sequentially on partial views of the videos. This reduces the computational complexity of our generative model, which scales to high-resolution videos beyond a few frames. We validate our approach on Kinetics-600 and BDD100K, for which we train a three level model capable of generating 256x256 videos with 48 frames.
References in corpus (21)
- Adam: A Method for Stochastic Optimization
- Language Models are Few-Shot Learners
- The Kinetics Human Action Video Dataset
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Unsupervised Learning of Video Representations using LSTMs
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Density estimation using Real NVP
- Adversarial Feature Learning
- A Recurrent Latent Variable Model for Sequential Data
- Adversarially Learned Inference
- Autoencoding beyond pixels using a learned similarity metric
- Decomposing Motion and Content for Natural Video Sequence Prediction
- Deep multi-scale video prediction beyond mean square error
- Stochastic Video Generation with a Learned Prior
- A Short Note about Kinetics-600
- Video (language) modeling: a baseline for generative models of natural videos
- Stochastic Adversarial Video Prediction
- Delving Deeper into Convolutional Networks for Learning Video Representations
- Adversarial Video Generation on Complex Datasets
- Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN
- Stochastic Variational Video Prediction