Causal Future Prediction in a Minkowski Space-Time
arXiv:2008.09154
Abstract
Estimating future events is a difficult task. Unlike humans, machine learning approaches are not regularized by a natural understanding of physics. In the wild, a plausible succession of events is governed by the rules of causality, which cannot easily be derived from a finite training set. In this paper we propose a novel theoretical framework to perform causal future prediction by embedding spatiotemporal information on a Minkowski space-time. We utilize the concept of a light cone from special relativity to restrict and traverse the latent space of an arbitrary model. We demonstrate successful applications in causal image synthesis and future video frame prediction on a dataset of images. Our framework is architecture- and task-independent and comes with strong theoretical guarantees of causal capabilities.
arXiv commentIncludes supplement
References in corpus (5)
- Unsupervised Learning of Video Representations using LSTMs
- Decomposing Motion and Content for Natural Video Sequence Prediction
- Learning Plannable Representations with Causal InfoGAN
- Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders
- Time-Agnostic Prediction: Predicting Predictable Video Frames