Improving Video Generation for Multi-functional Applications
arXiv:1711.11453
Abstract
In this paper, we aim to improve the state-of-the-art video generative adversarial networks (GANs) with a view towards multi-functional applications. Our improved video GAN model does not separate foreground from background nor dynamic from static patterns, but learns to generate the entire video clip conjointly. Our model can thus be trained to generate - and learn from - a broad set of videos with no restriction. This is achieved by designing a robust one-stream video generation architecture with an extension of the state-of-the-art Wasserstein GAN framework that allows for better convergence. The experimental results show that our improved video GAN model outperforms state-of-theart video generative models on multiple challenging datasets. Furthermore, we demonstrate the superiority of our model by successfully extending it to three challenging problems: video colorization, video inpainting, and future prediction. To the best of our knowledge, this is the first work using GANs to colorize and inpaint video clips.
References in corpus (20)
- Adam: A Method for Stochastic Optimization
- Conditional Generative Adversarial Nets
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Empirical Evaluation of Rectified Activations in Convolutional Network
- Densely Connected Convolutional Networks
- Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- YFCC100M: The New Data in Multimedia Research
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Energy-based Generative Adversarial Network
- Generating Videos with Scene Dynamics
- Video (language) modeling: a baseline for generative models of natural videos
- Unsupervised Learning for Physical Interaction through Video Prediction
- Mode Regularized Generative Adversarial Networks
- Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
- YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video
- High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis
- Video Pixel Networks
- Forecasting Human Dynamics from Static Images
- A learning-based approach for automatic image and video colorization