Fast Scene Understanding for Autonomous Driving

Abstract

Most approaches for instance-aware semantic labeling traditionally focus on accuracy. Other aspects like runtime and memory footprint are arguably as important for real-time applications such as autonomous driving. Motivated by this observation and inspired by recent works that tackle multiple tasks with a single integrated architecture, in this paper we present a real-time efficient implementation based on ENet that solves three autonomous driving related tasks at once: semantic scene segmentation, instance segmentation and monocular depth estimation. Our approach builds upon a branched ENet architecture with a shared encoder but different decoder branches for each of the three tasks. The presented method can run at 21 fps at a resolution of 1024x512 on the Cityscapes dataset without sacrificing accuracy compared to running each task separately.

Published at "Deep Learning for Vehicle Perception", workshop at the IEEE Symposium on Intelligent Vehicles 2017

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Deep Watershed Transform for Instance Segmentation
• InstanceCut: from Edges to Instances with MultiCut
• Boundary-aware Instance Segmentation

Cited by in corpus (14)

• Multi-Task Learning for Dense Prediction Tasks: A Survey
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• Learning to Fuse Things and Stuff
• RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentation
• Multiple-Human Parsing in the Wild
• A Hybrid Spatial-temporal Deep Learning Architecture for Lane Detection
• Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation
• Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Ability
• Regularizing Deep Multi-Task Networks using Orthogonal Gradients
• AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving
• Automated Bridge Component Recognition using Video Data
• RaidaR: A Rich Annotated Image Dataset of Rainy Street Scenes
• Monocular Outdoor Semantic Mapping with a Multi-task Network
• Spatial Sampling Network for Fast Scene Understanding