Flow-based Spatio-Temporal Structured Prediction of Motion Dynamics

Abstract

Conditional Normalizing Flows (CNFs) are flexible generative models capable of representing complicated distributions with high dimensionality and large interdimensional correlations, making them appealing for structured output learning. Their effectiveness in modelling multivariates spatio-temporal structured data has yet to be completely investigated. We propose MotionFlow as a novel normalizing flows approach that autoregressively conditions the output distributions on the spatio-temporal input features. It combines deterministic and stochastic representations with CNFs to create a probabilistic neural generative approach that can model the variability seen in high dimensional structured spatio-temporal data. We specifically propose to use conditional priors to factorize the latent space for the time dependent modeling. We also exploit the use of masked convolutions as autoregressive conditionals in CNFs. As a result, our method is able to define arbitrarily expressive output probability distributions under temporal dynamics in multivariate prediction tasks. We apply our method to different tasks, including trajectory prediction, motion prediction, time series forecasting, and binary segmentation, and demonstrate that our model is able to leverage normalizing flows to learn complicated time dependent conditional distributions.

arXiv comment13 pages, LaTeX; typos corrected, updated, in IEEE Transactions on Pattern Analysis and Machine Intelligence

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Neural Machine Translation by Jointly Learning to Align and Translate
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Normalizing Flows: An Introduction and Review of Current Methods
• Neural Relational Inference for Interacting Systems
• Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
• EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning
• SparseMAP: Differentiable Sparse Structured Inference
• Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows
• Learning Approximate Inference Networks for Structured Prediction
• Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs
• Conditional Density Estimation with Bayesian Normalising Flows
• Semi-Conditional Normalizing Flows for Semi-Supervised Learning
• Graphical Normalizing Flows
• Interaction Modeling with Multiplex Attention
• Locally Masked Convolution for Autoregressive Models