Deep Structured Reactive Planning

Abstract

An intelligent agent operating in the real-world must balance achieving its goal with maintaining the safety and comfort of not only itself, but also other participants within the surrounding scene. This requires jointly reasoning about the behavior of other actors while deciding its own actions as these two processes are inherently intertwined - a vehicle will yield to us if we decide to proceed first at the intersection but will proceed first if we decide to yield. However, this is not captured in most self-driving pipelines, where planning follows prediction. In this paper we propose a novel data-driven, reactive planning objective which allows a self-driving vehicle to jointly reason about its own plans as well as how other actors will react to them. We formulate the problem as an energy-based deep structured model that is learned from observational data and encodes both the planning and prediction problems. Through simulations based on both real-world driving and synthetically generated dense traffic, we demonstrate that our reactive model outperforms a non-reactive variant in successfully completing highly complex maneuvers (lane merges/turns in traffic) faster, without trading off collision rate.

arXiv commentICRA 2021

References in corpus (17)

• End to End Learning for Self-Driving Cars
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Fully Connected Deep Structured Networks
• IntentNet: Learning to Predict Intention from Raw Sensor Data
• Baidu Apollo EM Motion Planner
• Driving Policy Transfer via Modularity and Abstraction
• Multiple Futures Prediction
• ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
• DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
• Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
• VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation
• End-to-End Learning for Structured Prediction Energy Networks
• Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions
• DSDNet: Deep Structured self-Driving Network
• PnPNet: End-to-End Perception and Prediction with Tracking in the Loop
• Amortized Bethe Free Energy Minimization for Learning MRFs

Cited by in corpus (1)

• Scene Transformer: A unified architecture for predicting multiple agent trajectories