IntentNet: Learning to Predict Intention from Raw Sensor Data

Abstract

In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants. We define intent as a combination of discrete high-level behaviors as well as continuous trajectories describing future motion. In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.

CoRL 2018

Cited by in corpus (76)

• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review
• A Survey of End-to-End Driving: Architectures and Training Methods
• TNT: Target-driveN Trajectory Prediction
• A Review of Tracking, Prediction and Decision Making Methods for Autonomous Driving
• Argoverse: 3D Tracking and Forecasting with Rich Maps
• MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction
• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
• Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• Occupancy Flow Fields for Motion Forecasting in Autonomous Driving
• SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks
• Learning to Predict Vehicle Trajectories with Model-based Planning
• Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
• VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation
• 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
• Learning Lane Graph Representations for Motion Forecasting
• BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud
• Map-Adaptive Goal-Based Trajectory Prediction
• Joint Interaction and Trajectory Prediction for Autonomous Driving using Graph Neural Networks
• Auto4D: Learning to Label 4D Objects from Sequential Point Clouds
• What-If Motion Prediction for Autonomous Driving
• Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions
• Deep Learning for Vision-based Prediction: A Survey
• DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets
• Scenario-Transferable Semantic Graph Reasoning for Interaction-Aware Probabilistic Prediction
• PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous driving
• PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling
• V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction
• FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras
• Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks
• Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction
• Robust Multi-Modality Multi-Object Tracking
• DSDNet: Deep Structured self-Driving Network
• Relation-based Motion Prediction using Traffic Scene Graphs
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion
• Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehicles
• An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds
• MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps
• STINet: Spatio-Temporal-Interactive Network for Pedestrian Detection and Trajectory Prediction
• AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
• PePScenes: A Novel Dataset and Baseline for Pedestrian Action Prediction in 3D
• StrObe: Streaming Object Detection from LiDAR Packets
• PolarStream: Streaming Lidar Object Detection and Segmentation with Polar Pillars
• PRANK: motion Prediction based on RANKing
• PnPNet: End-to-End Perception and Prediction with Tracking in the Loop
• Physically Feasible Vehicle Trajectory Prediction
• Planning Robot Motion using Deep Visual Prediction
• Deep Kinematic Models for Kinematically Feasible Vehicle Trajectory Predictions
• TPCN: Temporal Point Cloud Networks for Motion Forecasting
• Motion Prediction using Trajectory Sets and Self-Driving Domain Knowledge
• Motion Prediction on Self-driving Cars: A Review
• Collaborative Motion Prediction via Neural Motion Message Passing
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
• End-to-end Interpretable Neural Motion Planner
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors
• Exploring Dynamic Context for Multi-path Trajectory Prediction
• Investigating the Effect of Sensor Modalities in Multi-Sensor Detection-Prediction Models
• Imitative Planning using Conditional Normalizing Flow
• Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection
• Uncertainty-Aware Vehicle Orientation Estimation for Joint Detection-Prediction Models
• Universal Embeddings for Spatio-Temporal Tagging of Self-Driving Logs
• ParkPredict: Motion and Intent Prediction of Vehicles in Parking Lots
• Ellipse Loss for Scene-Compliant Motion Prediction
• Iterative Imitation Policy Improvement for Interactive Autonomous Driving
• Towards Robust Human Trajectory Prediction in Raw Videos
• Deep Structured Reactive Planning
• Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes
• Long-term Prediction of Vehicle Behavior using Short-term Uncertainty-aware Trajectories and High-definition Maps
• Convolutional Recurrent Network for Road Boundary Extraction
• Open-set Intersection Intention Prediction for Autonomous Driving
• Motion Prediction Performance Analysis for Autonomous Driving Systems and the Effects of Tracking Noise
• Convolutions for Spatial Interaction Modeling
• Drivers' Manoeuvre Modelling and Prediction for Safe HRI
• Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning
• DAGMapper: Learning to Map by Discovering Lane Topology