Multiple Futures Prediction

Abstract

Temporal prediction is critical for making intelligent and robust decisions in complex dynamic environments. Motion prediction needs to model the inherently uncertain future which often contains multiple potential outcomes, due to multi-agent interactions and the latent goals of others. Towards these goals, we introduce a probabilistic framework that efficiently learns latent variables to jointly model the multi-step future motions of agents in a scene. Our framework is data-driven and learns semantically meaningful latent variables to represent the multimodal future, without requiring explicit labels. Using a dynamic attention-based state encoder, we learn to encode the past as well as the future interactions among agents, efficiently scaling to any number of agents. Finally, our model can be used for planning via computing a conditional probability density over the trajectories of other agents given a hypothetical rollout of the 'self' agent. We demonstrate our algorithms by predicting vehicle trajectories of both simulated and real data, demonstrating the state-of-the-art results on several vehicle trajectory datasets.

In proceedings of NeurIPS 2019, Vancouver, British Columbia, Canada

Cited by in corpus (44)

• TNT: Target-driveN Trajectory Prediction
• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
• Injecting Knowledge in Data-driven Vehicle Trajectory Predictors
• A Novel Graph based Trajectory Predictor with Pseudo Oracle
• Learning to Predict Vehicle Trajectories with Model-based Planning
• THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling
• VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation
• What-If Motion Prediction for Autonomous Driving
• Multimodal Motion Prediction with Stacked Transformers
• DROGON: A Trajectory Prediction Model based on Intention-Conditioned Behavior Reasoning
• PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous driving
• FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras
• Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehicles
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion
• Deformation Driven Seq2Seq Longitudinal Tumor and Organs-at-Risk Prediction for Radiotherapy
• Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Prediction
• Multimodal Trajectory Prediction via Topological Invariance for Navigation at Uncontrolled Intersections
• LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents
• Scene Transformer: A unified architecture for predicting multiple agent trajectories
• TPPO: A Novel Trajectory Predictor with Pseudo Oracle
• Hierarchical Motion Encoder-Decoder Network for Trajectory Forecasting
• Motion Prediction using Trajectory Sets and Self-Driving Domain Knowledge
• Human Trajectory Prediction via Counterfactual Analysis
• Congestion-aware Multi-agent Trajectory Prediction for Collision Avoidance
• DiversityGAN: Diversity-Aware Vehicle Motion Prediction via Latent Semantic Sampling
• Exploring Dynamic Context for Multi-path Trajectory Prediction
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
• AMENet: Attentive Maps Encoder Network for Trajectory Prediction
• Multi-future Merchant Transaction Prediction
• Pylot: A Modular Platform for Exploring Latency-Accuracy Tradeoffs in Autonomous Vehicles
• Learning interaction rules from multi-animal trajectories via augmented behavioral models
• From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video
• Relational State-Space Model for Stochastic Multi-Object Systems
• Deep Structured Reactive Planning
• Recursive Least Squares Based Refinement Network for the Rollout Trajectory Prediction Methods
• Vehicle Trajectory Prediction by Transfer Learning of Semi-Supervised Models
• RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting
• Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes
• AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points
• Polynomial Trajectory Predictions for Improved Learning Performance
• Probability Trajectory: One New Movement Description for Trajectory Prediction
• Consistent Multiple Sequence Decoding
• Planning on a (Risk) Budget: Safe Non-Conservative Planning in Probabilistic Dynamic Environments
• Handwriting Prediction Considering Inter-Class Bifurcation Structures