Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Ability

Abstract

Current end-to-end deep learning driving models have two problems: (1) Poor generalization ability of unobserved driving environment when diversity of training driving dataset is limited (2) Lack of accident explanation ability when driving models don't work as expected. To tackle these two problems, rooted on the believe that knowledge of associated easy task is benificial for addressing difficult task, we proposed a new driving model which is composed of perception module for \textit{see and think} and driving module for \textit{behave}, and trained it with multi-task perception-related basic knowledge and driving knowledge stepwisely. Specifically segmentation map and depth map (pixel level understanding of images) were considered as \textit{what \& where} and \textit{how far} knowledge for tackling easier driving-related perception problems before generating final control commands for difficult driving task. The results of experiments demonstrated the effectiveness of multi-task perception knowledge for better generalization and accident explanation ability. With our method the average sucess rate of finishing most difficult navigation tasks in untrained city of CoRL test surpassed current benchmark method for 15 percent in trained weather and 20 percent in untrained weathers. Demonstration video link is: https://www.youtube.com/watch?v=N7ePnnZZwdE

11 pages, 4 figures, 3 tables

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• End to End Learning for Self-Driving Cars
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• SmoothGrad: removing noise by adding noise
• Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
• CARLA: An Open Urban Driving Simulator
• DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
• Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues
• Fast Scene Understanding for Autonomous Driving
• VisualBackProp: efficient visualization of CNNs

Cited by in corpus (8)

• Multimodal End-to-End Autonomous Driving
• Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding
• Deep Imitative Models for Flexible Inference, Planning, and Control
• SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning
• lamBERT: Language and Action Learning Using Multimodal BERT
• Learning hierarchical behavior and motion planning for autonomous driving
• Multi-task Learning with Attention for End-to-end Autonomous Driving
• Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory