PDE-Driven Spatiotemporal Disentanglement

Abstract

A recent line of work in the machine learning community addresses the problem of predicting high-dimensional spatiotemporal phenomena by leveraging specific tools from the differential equations theory. Following this direction, we propose in this article a novel and general paradigm for this task based on a resolution method for partial differential equations: the separation of variables. This inspiration allows us to introduce a dynamical interpretation of spatiotemporal disentanglement. It induces a principled model based on learning disentangled spatial and temporal representations of a phenomenon to accurately predict future observations. We experimentally demonstrate the performance and broad applicability of our method against prior state-of-the-art models on physical and synthetic video datasets.

References in corpus (21)

• Auto-Encoding Variational Bayes
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Estimating Mutual Information
• DGM: A deep learning algorithm for solving partial differential equations
• Unsupervised Learning of Video Representations using LSTMs
• Mixed Precision Training
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Hamiltonian Neural Networks
• Decomposing Motion and Content for Natural Video Sequence Prediction
• PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
• Stochastic Video Generation with a Learned Prior
• Accelerating Eulerian Fluid Simulation With Convolutional Networks
• Stochastic Adversarial Video Prediction
• Scalable Gradients for Stochastic Differential Equations
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• ODE${}^2$VAE: Deep generative second order ODEs with Bayesian neural networks
• Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
• Disentangled Sequential Autoencoder
• Stochastic Latent Residual Video Prediction
• Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video

Cited by in corpus (3)

• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting
• Guiding Video Prediction with Explicit Procedural Knowledge
• Gaussian Process Latent Variable Modeling for Few-shot Time Series Forecasting