Dynamic Variational Autoencoders for Visual Process Modeling

Abstract

This work studies the problem of modeling visual processes by leveraging deep generative architectures for learning linear, Gaussian representations from observed sequences. We propose a joint learning framework, combining a vector autoregressive model and Variational Autoencoders. This results in an architecture that allows Variational Autoencoders to simultaneously learn a non-linear observation as well as a linear state model from sequences of frames. We validate our approach on artificial sequences and dynamic textures.

References in corpus (15)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Unsupervised Learning of Video Representations using LSTMs
• Tutorial on Variational Autoencoders
• Generating Videos with Scene Dynamics
• Understanding Deep Convolutional Networks
• Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
• Composing graphical models with neural networks for structured representations and fast inference
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
• Deep Kalman Filters
• Transformation Properties of Learned Visual Representations
• Deep Predictive Coding Networks
• Fault-Tolerant Control of Linear Quantum Stochastic Systems
• Learning Dynamic Generator Model by Alternating Back-Propagation Through Time
• Alignment Distances on Systems of Bags