Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data

Abstract

In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor states via a message passing process. Inspired by Gaussian belief propagation, we design the messages to be spatially-transformed parameters of the output distributions from neighboring agents. Our model is fully differentiable, thus enabling end-to-end training. Importantly, our probabilistic predictions can model uncertainty at the trajectory level. We demonstrate the effectiveness of our approach by achieving significant improvements over the state-of-the-art on two real-world self-driving datasets: ATG4D and nuScenes.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• Vehicle Detection from 3D Lidar Using Fully Convolutional Network
• IntentNet: Learning to Predict Intention from Raw Sensor Data
• Neural Relational Inference for Interacting Systems
• HDNET: Exploiting HD Maps for 3D Object Detection
• Complex-YOLO: Real-time 3D Object Detection on Point Clouds
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud
• Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving

Cited by in corpus (19)

• One Thousand and One Hours: Self-driving Motion Prediction Dataset
• Learning Lane Graph Representations for Motion Forecasting
• Boost then Convolve: Gradient Boosting Meets Graph Neural Networks
• V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction
• A PAC-Bayesian Approach to Generalization Bounds for Graph Neural Networks
• DSDNet: Deep Structured self-Driving Network
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion
• Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
• Physically Feasible Vehicle Trajectory Prediction
• Context-aware demand prediction in bike sharing systems: incorporating spatial, meteorological and calendrical context
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
• Imitative Planning using Conditional Normalizing Flow
• Auditing AI models for Verified Deployment under Semantic Specifications
• Universal Embeddings for Spatio-Temporal Tagging of Self-Driving Logs
• Deep Structured Reactive Planning
• Vehicle Trajectory Prediction by Transfer Learning of Semi-Supervised Models
• Ellipse Loss for Scene-Compliant Motion Prediction
• LTN: Long-Term Network for Long-Term Motion Prediction
• Convolutions for Spatial Interaction Modeling