Improving Video Generation for Multi-functional Applications
arXiv:1711.11453
Abstract
In this paper, we aim to improve the state-of-the-art video generative adversarial networks (GANs) with a view towards multi-functional applications. Our improved video GAN model does not separate foreground from background nor dynamic from static patterns, but learns to generate the entire video clip conjointly. Our model can thus be trained to generate - and learn from - a broad set of videos with no restriction. This is achieved by designing a robust one-stream video generation architecture with an extension of the state-of-the-art Wasserstein GAN framework that allows for better convergence. The experimental results show that our improved video GAN model outperforms state-of-theart video generative models on multiple challenging datasets. Furthermore, we demonstrate the superiority of our model by successfully extending it to three challenging problems: video colorization, video inpainting, and future prediction. To the best of our knowledge, this is the first work using GANs to colorize and inpaint video clips.
References in corpus (12)
- Conditional Generative Adversarial Nets
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Energy-based Generative Adversarial Network
- Generating Videos with Scene Dynamics
- Mode Regularized Generative Adversarial Networks
- Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
- YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video
- Video Pixel Networks
- Forecasting Human Dynamics from Static Images
- A learning-based approach for automatic image and video colorization