Video Extrapolation with an Invertible Linear Embedding
arXiv:1903.00133
Abstract
We predict future video frames from complex dynamic scenes, using an invertible neural network as the encoder of a nonlinear dynamic system with latent linear state evolution. Our invertible linear embedding (ILE) demonstrates successful learning, prediction and latent state inference. In contrast to other approaches, ILE does not use any explicit reconstruction loss or simplistic pixel-space assumptions. Instead, it leverages invertibility to optimize the likelihood of image sequences exactly, albeit indirectly. Comparison with a state-of-the-art method demonstrates the viability of our approach.
References in corpus (11)
- Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
- Deep learning for universal linear embeddings of nonlinear dynamics
- NICE: Non-linear Independent Components Estimation
- Density estimation using Real NVP
- Action-Conditional Video Prediction using Deep Networks in Atari Games
- Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
- Deep multi-scale video prediction beyond mean square error
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- Recurrent Environment Simulators
- Robust Locally-Linear Controllable Embedding
- A Dataset To Evaluate The Representations Learned By Video Prediction Models