Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes

Abstract

Semantic image segmentation is an essential component of modern autonomous driving systems, as an accurate understanding of the surrounding scene is crucial to navigation and action planning. Current state-of-the-art approaches in semantic image segmentation rely on pre-trained networks that were initially developed for classifying images as a whole. While these networks exhibit outstanding recognition performance (i.e., what is visible?), they lack localization accuracy (i.e., where precisely is something located?). Therefore, additional processing steps have to be performed in order to obtain pixel-accurate segmentation masks at the full image resolution. To alleviate this problem we propose a novel ResNet-like architecture that exhibits strong localization and recognition performance. We combine multi-scale context with pixel-level accuracy by using two processing streams within our network: One stream carries information at the full image resolution, enabling precise adherence to segment boundaries. The other stream undergoes a sequence of pooling operations to obtain robust features for recognition. The two streams are coupled at the full image resolution using residuals. Without additional processing steps and without pre-training, our approach achieves an intersection-over-union score of 71.8% on the Cityscapes dataset.

arXiv commentChanges in v2: Fixed equation (10), fixed legend of Figure 6, fixed legend of Figure 9, added page numbers, fixed minor spelling mistakes

References in corpus (26)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Residual Learning for Image Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Conditional Random Fields as Recurrent Neural Networks
• Multi-Scale Context Aggregation by Dilated Convolutions
• Going Deeper with Convolutions
• ParseNet: Looking Wider to See Better
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Fully Convolutional Networks for Semantic Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Learning Deconvolution Network for Semantic Segmentation
• Stacked Hourglass Networks for Human Pose Estimation
• Training Deep Nets with Sublinear Memory Cost
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Fully Connected Deep Structured Networks
• Simultaneous Detection and Segmentation
• Semantic Image Segmentation via Deep Parsing Network
• Bridging Category-level and Instance-level Semantic Image Segmentation
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation
• segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection
• Memory-Efficient Backpropagation Through Time
• Feedforward semantic segmentation with zoom-out features

Cited by in corpus (7)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation
• ExFuse: Enhancing Feature Fusion for Semantic Segmentation
• Fully Convolutional Networks for Chip-wise Defect Detection Employing Photoluminescence Images
• DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
• BusyHands: A Hand-Tool Interaction Database for Assembly Tasks Semantic Segmentation
• EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation