Driving Scene Perception Network: Real-time Joint Detection, Depth Estimation and Semantic Segmentation

Abstract

As the demand for enabling high-level autonomous driving has increased in recent years and visual perception is one of the critical features to enable fully autonomous driving, in this paper, we introduce an efficient approach for simultaneous object detection, depth estimation and pixel-level semantic segmentation using a shared convolutional architecture. The proposed network model, which we named Driving Scene Perception Network (DSPNet), uses multi-level feature maps and multi-task learning to improve the accuracy and efficiency of object detection, depth estimation and image segmentation tasks from a single input image. Hence, the resulting network model uses less than 850 MiB of GPU memory and achieves 14.0 fps on NVIDIA GeForce GTX 1080 with a 1024x512 input image, and both precision and efficiency have been improved over combination of single tasks.

arXiv comment9 pages, 7 figures, WACV'18

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Going Deeper with Convolutions
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Visualizing and Understanding Convolutional Networks
• An Empirical Evaluation of Deep Learning on Highway Driving
• Xception: Deep Learning with Depthwise Separable Convolutions
• Pyramid Scene Parsing Network

Cited by in corpus (7)

• Towards Compact Autonomous Driving Perception with Balanced Learning and Multi-sensor Fusion
• AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving
• Context-Integrated and Feature-Refined Network for Lightweight Object Parsing
• Multi-Task Semantic Communication With Graph Attention-Based Feature Correlation Extraction
• Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation
• Your "Flamingo" is My "Bird": Fine-Grained, or Not
• Deep Joint Source-Channel Coding for Multi-Task Network