PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling

Abstract

Multi-object tracking (MOT) and trajectory prediction are two critical components in modern 3D perception systems that require accurate modeling of multi-agent interaction. We hypothesize that it is beneficial to unify both tasks under one framework in order to learn a shared feature representation of agent interaction. Furthermore, instead of performing tracking and prediction sequentially which can propagate errors from tracking to prediction, we propose a parallelized framework to mitigate the issue. Also, our parallel track-forecast framework incorporates two additional novel computational units. First, we use a feature interaction technique by introducing Graph Neural Networks (GNNs) to capture the way in which agents interact with one another. The GNN is able to improve discriminative feature learning for MOT association and provide socially-aware contexts for trajectory prediction. Second, we use a diversity sampling function to improve the quality and diversity of our forecasted trajectories. The learned sampling function is trained to efficiently extract a variety of outcomes from a generative trajectory distribution and helps avoid the problem of generating duplicate trajectory samples. We evaluate on KITTI and nuScenes datasets showing that our method with socially-aware feature learning and diversity sampling achieves new state-of-the-art performance on 3D MOT and trajectory prediction. Project website is: https://www.xinshuoweng.com/projects/PTP

arXiv commentPublished in Robotics and Automation Letters (RA-L) 2021, with the ICRA 2021 option. The first two authors contributed equally. Project website: https://www.xinshuoweng.com/projects/PTP/

References in corpus (12)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• MOT16: A Benchmark for Multi-Object Tracking
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
• IntentNet: Learning to Predict Intention from Raw Sensor Data
• Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
• End-to-end Learning of Multi-sensor 3D Tracking by Detection
• Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
• Diverse Trajectory Forecasting with Determinantal Point Processes
• Deep Reinforcement Learning for Autonomous Driving
• Expectation-Maximization for Learning Determinantal Point Processes

Cited by in corpus (9)

• AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics
• 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
• TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking
• GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning
• DLow: Diversifying Latent Flows for Diverse Human Motion Prediction
• FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
• Generative Hybrid Representations for Activity Forecasting with No-Regret Learning
• When We First Met: Visual-Inertial Person Localization for Co-Robot Rendezvous
• AutoSelect: Automatic and Dynamic Detection Selection for 3D Multi-Object Tracking