Stochastic Prediction of Multi-Agent Interactions from Partial Observations

Abstract

We present a method that learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents. Our method is based on a graph-structured variational recurrent neural network (Graph-VRNN), which is trained end-to-end to infer the current state of the (partially observed) world, as well as to forecast future states. We show that our method outperforms various baselines on two sports datasets, one based on real basketball trajectories, and one generated by a soccer game engine.

ICLR 2019 camera ready

Cited by in corpus (17)

• Learning to Simulate Complex Physics with Graph Networks
• Object-Centric Learning with Slot Attention
• On the Binding Problem in Artificial Neural Networks
• VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation
• AC-VRNN: Attentive Conditional-VRNN for Multi-Future Trajectory Prediction
• Multi-agent Trajectory Prediction with Fuzzy Query Attention
• Unsupervised Learning of Object Structure and Dynamics from Videos
• Scenario-Transferable Semantic Graph Reasoning for Interaction-Aware Probabilistic Prediction
• SoDA: Multi-Object Tracking with Soft Data Association
• CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
• Learning Object-Centric Video Models by Contrasting Sets
• From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video
• Learning Temporal Dynamics from Cycles in Narrated Video
• Relational State-Space Model for Stochastic Multi-Object Systems
• Deep Variational Luenberger-type Observer for Stochastic Video Prediction
• Consistent Multiple Sequence Decoding
• Graph-Based Social Relation Reasoning