Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data

Abstract

In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor states via a message passing process. Inspired by Gaussian belief propagation, we design the messages to be spatially-transformed parameters of the output distributions from neighboring agents. Our model is fully differentiable, thus enabling end-to-end training. Importantly, our probabilistic predictions can model uncertainty at the trajectory level. We demonstrate the effectiveness of our approach by achieving significant improvements over the state-of-the-art on two real-world self-driving datasets: ATG4D and nuScenes.

References in corpus (5)

• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• Vehicle Detection from 3D Lidar Using Fully Convolutional Network
• IntentNet: Learning to Predict Intention from Raw Sensor Data
• HDNET: Exploiting HD Maps for 3D Object Detection
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud

Cited by in corpus (10)

• Learning Lane Graph Representations for Motion Forecasting
• Boost then Convolve: Gradient Boosting Meets Graph Neural Networks
• V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction
• A PAC-Bayesian Approach to Generalization Bounds for Graph Neural Networks
• DSDNet: Deep Structured self-Driving Network
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
• Auditing AI models for Verified Deployment under Semantic Specifications
• Universal Embeddings for Spatio-Temporal Tagging of Self-Driving Logs
• Ellipse Loss for Scene-Compliant Motion Prediction