Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation

Abstract

Recently, deep-learning based approaches have achieved impressive performance for autonomous driving. However, end-to-end vision-based methods typically have limited interpretability, making the behaviors of the deep networks difficult to explain. Hence, their potential applications could be limited in practice. To address this problem, we propose an interpretable end-to-end vision-based motion planning approach for autonomous driving, referred to as IVMP. Given a set of past surrounding-view images, our IVMP first predicts future egocentric semantic maps in bird's-eye-view space, which are then employed to plan trajectories for self-driving vehicles. The predicted future semantic maps not only provide useful interpretable information, but also allow our motion planning module to handle objects with low probability, thus improving the safety of autonomous driving. Moreover, we also develop an optical flow distillation paradigm, which can effectively enhance the network while still maintaining its real-time performance. Extensive experiments on the nuScenes dataset and closed-loop simulation show that our IVMP significantly outperforms the state-of-the-art approaches in imitating human drivers with a much higher success rate. Our project page is available at https://sites.google.com/view/ivmp.

arXiv comment7 pages, 5 figures and 1 table. This paper is accepted by ICRA 2021. arXiv admin note: text overlap with arXiv:2104.08862

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• End to End Learning for Self-Driving Cars
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection
• Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks
• Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras
• Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms
• Learning Collision-Free Space Detection from Stereo Images: Homography Matrix Brings Better Data Augmentation