Machine Learning for Spatiotemporal Sequence Forecasting: A Survey

Abstract

Spatiotemporal systems are common in the real-world. Forecasting the multi-step future of these spatiotemporal systems based on the past observations, or, Spatiotemporal Sequence Forecasting (STSF), is a significant and challenging problem. Although lots of real-world problems can be viewed as STSF and many research works have proposed machine learning based methods for them, no existing work has summarized and compared these methods from a unified perspective. This survey aims to provide a systematic review of machine learning for STSF. In this survey, we define the STSF problem and classify it into three subcategories: Trajectory Forecasting of Moving Point Cloud (TF-MPC), STSF on Regular Grid (STSF-RG) and STSF on Irregular Grid (STSF-IG). We then introduce the two major challenges of STSF: 1) how to learn a model for multi-step forecasting and 2) how to adequately model the spatial and temporal structures. After that, we review the existing works for solving these challenges, including the general learning strategies for multi-step forecasting, the classical machine learning based methods for STSF, and the deep learning based methods for STSF. We also compare these methods and point out some potential research directions.

References in corpus (19)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Inductive Representation Learning on Large Graphs
• WaveNet: A Generative Model for Raw Audio
• Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Inferring causal impact using Bayesian structural time-series models
• Generating Videos with Scene Dynamics
• Decomposing Motion and Content for Natural Video Sequence Prediction
• PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Stochastic Video Generation with a Learned Prior
• Video (language) modeling: a baseline for generative models of natural videos
• GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• Recurrent Network Models for Human Dynamics
• Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective
• Video Pixel Networks

Cited by in corpus (11)

• Graph Neural Network for Traffic Forecasting: A Survey
• Bayesian Temporal Factorization for Multidimensional Time Series Prediction
• SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction
• ETA Prediction with Graph Neural Networks in Google Maps
• A review of radar-based nowcasting of precipitation and applicable machine learning techniques
• STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems
• DFSeer: A Visual Analytics Approach to Facilitate Model Selection for Demand Forecasting
• Localized convolutional neural networks for geospatial wind forecasting
• CAMETA: Conflict-Aware Multi-Agent Estimated Time of Arrival Prediction for Mobile Robots
• BU-Trace: A Permissionless Mobile System for Privacy-Preserving Intelligent Contact Tracing
• Bridging the Gap Between Training and Inference for Spatio-Temporal Forecasting