Learning to navigate image manifolds induced by generative adversarial networks for unsupervised video generation

Abstract

In this work, we introduce a two-step framework for generative modeling of temporal data. Specifically, the generative adversarial networks (GANs) setting is employed to generate synthetic scenes of moving objects. To do so, we propose a two-step training scheme within which: a generator of static frames is trained first. Afterwards, a recurrent model is trained with the goal of providing a sequence of inputs to the previously trained frames generator, thus yielding scenes which look natural. The adversarial setting is employed in both training steps. However, with the aim of avoiding known training instabilities in GANs, a multiple discriminator approach is used to train both models. Results in the studied video dataset indicate that, by employing such an approach, the recurrent part is able to learn how to coherently navigate the image manifold induced by the frames generator, thus yielding more natural-looking scenes.

References in corpus (7)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Unsupervised Learning of Video Representations using LSTMs
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• NIPS 2016 Tutorial: Generative Adversarial Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Stabilizing GAN Training with Multiple Random Projections
• Attacking Speaker Recognition With Deep Generative Models