TNT: Target-driveN Trajectory Prediction

Abstract

Predicting the future behavior of moving agents is essential for real world applications. It is challenging as the intent of the agent and the corresponding behavior is unknown and intrinsically multimodal. Our key insight is that for prediction within a moderate time horizon, the future modes can be effectively captured by a set of target states. This leads to our target-driven trajectory prediction (TNT) framework. TNT has three stages which are trained end-to-end. It first predicts an agent's potential target states $T$ steps into the future, by encoding its interactions with the environment and the other agents. TNT then generates trajectory state sequences conditioned on targets. A final stage estimates trajectory likelihoods and a final compact set of trajectory predictions is selected. This is in contrast to previous work which models agent intents as latent variables, and relies on test-time sampling to generate diverse trajectories. We benchmark TNT on trajectory prediction of vehicles and pedestrians, where we outperform state-of-the-art on Argoverse Forecasting, INTERACTION, Stanford Drone and an in-house Pedestrian-at-Intersection dataset.

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Social Force Model for Pedestrian Dynamics
• Variational Inference with Normalizing Flows
• INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps
• IntentNet: Learning to Predict Intention from Raw Sensor Data
• Argoverse: 3D Tracking and Forecasting with Rich Maps
• Multiple Futures Prediction
• Generating Long-term Trajectories Using Deep Hierarchical Networks
• Generating Multi-Agent Trajectories using Programmatic Weak Supervision
• Forecasting Social Navigation in Crowded Complex Scenes

Cited by in corpus (31)

• Stepwise Goal-Driven Networks for Trajectory Prediction
• A Review of Tracking, Prediction and Decision Making Methods for Autonomous Driving
• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
• Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans
• Learning to Predict Vehicle Trajectories with Model-based Planning
• THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling
• From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting
• Path-Aware Graph Attention for HD Maps in Motion Prediction
• Multimodal Motion Prediction with Stacked Transformers
• DROGON: A Trajectory Prediction Model based on Intention-Conditioned Behavior Reasoning
• DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets
• Heterogeneous Edge-Enhanced Graph Attention Network For Multi-Agent Trajectory Prediction
• Scenario-Transferable Semantic Graph Reasoning for Interaction-Aware Probabilistic Prediction
• DenseTNT: Waymo Open Dataset Motion Prediction Challenge 1st Place Solution
• Comprehensive Reactive Safety: No Need For A Trajectory If You Have A Strategy
• Relation-based Motion Prediction using Traffic Scene Graphs
• Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Prediction
• LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents
• Scene Transformer: A unified architecture for predicting multiple agent trajectories
• TPCN: Temporal Point Cloud Networks for Motion Forecasting
• Physically Feasible Vehicle Trajectory Prediction
• Reimagining an autonomous vehicle
• Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction
• Graph and Recurrent Neural Network-based Vehicle Trajectory Prediction For Highway Driving
• Interpretable and Flexible Target-Conditioned Neural Planners For Autonomous Vehicles
• Trajectory Prediction with Graph-based Dual-scale Context Fusion
• Prediction-Based Reachability for Collision Avoidance in Autonomous Driving
• Collaborative Uncertainty in Multi-Agent Trajectory Forecasting
• Jointly Learning Agent and Lane Information for Multimodal Trajectory Prediction
• CRITERIA: a New Benchmarking Paradigm for Evaluating Trajectory Prediction Models for Autonomous Driving
• On complementing end-to-end human behavior predictors with planning