TPCN: Temporal Point Cloud Networks for Motion Forecasting

Abstract

We propose the Temporal Point Cloud Networks (TPCN), a novel and flexible framework with joint spatial and temporal learning for trajectory prediction. Unlike existing approaches that rasterize agents and map information as 2D images or operate in a graph representation, our approach extends ideas from point cloud learning with dynamic temporal learning to capture both spatial and temporal information by splitting trajectory prediction into both spatial and temporal dimensions. In the spatial dimension, agents can be viewed as an unordered point set, and thus it is straightforward to apply point cloud learning techniques to model agents' locations. While the spatial dimension does not take kinematic and motion information into account, we further propose dynamic temporal learning to model agents' motion over time. Experiments on the Argoverse motion forecasting benchmark show that our approach achieves the state-of-the-art results.

accepted to CVPR 2021

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Convolutional Networks on Graph-Structured Data
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• IntentNet: Learning to Predict Intention from Raw Sensor Data
• TNT: Target-driveN Trajectory Prediction
• Argoverse: 3D Tracking and Forecasting with Rich Maps
• MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction
• Learning Lane Graph Representations for Motion Forecasting
• Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution