LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving

Abstract

In this paper, we present LookOut, a novel autonomy system that perceives the environment, predicts a diverse set of futures of how the scene might unroll and estimates the trajectory of the SDV by optimizing a set of contingency plans over these future realizations. In particular, we learn a diverse joint distribution over multi-agent future trajectories in a traffic scene that covers a wide range of future modes with high sample efficiency while leveraging the expressive power of generative models. Unlike previous work in diverse motion forecasting, our diversity objective explicitly rewards sampling future scenarios that require distinct reactions from the self-driving vehicle for improved safety. Our contingency planner then finds comfortable and non-conservative trajectories that ensure safe reactions to a wide range of future scenarios. Through extensive evaluations, we show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a large-scale self-driving dataset as well as safer and less-conservative motion plans in long-term closed-loop simulations when compared to current state-of-the-art models.

References in corpus (7)

• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• IntentNet: Learning to Predict Intention from Raw Sensor Data
• HDNET: Exploiting HD Maps for 3D Object Detection
• TNT: Target-driveN Trajectory Prediction
• MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction
• Multiple Futures Prediction
• Decision-Time Postponing Motion Planning for Combinatorial Uncertain Maneuvering

Cited by in corpus (7)

• RACP: Risk-Aware Contingency Planning with Multi-Modal Predictions
• Efficient Speed Planning for Autonomous Driving in Dynamic Environment with Interaction Point Model
• Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios
• IR-STP: Enhancing Autonomous Driving with Interaction Reasoning in Spatio-Temporal Planning
• CASPFormer: Trajectory Prediction from BEV Images with Deformable Attention
• Curb Your Attention: Causal Attention Gating for Robust Trajectory Prediction in Autonomous Driving
• Parallel Neural Computing for Scene Understanding from LiDAR Perception in Autonomous Racing