High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks

Abstract

We propose a method for high-performance semantic image segmentation (or semantic pixel labelling) based on very deep residual networks, which achieves the state-of-the-art performance. A few design factors are carefully considered to this end. We make the following contributions. (i) First, we evaluate different variations of a fully convolutional residual network so as to find the best configuration, including the number of layers, the resolution of feature maps, and the size of field-of-view. Our experiments show that further enlarging the field-of-view and increasing the resolution of feature maps are typically beneficial, which however inevitably leads to a higher demand for GPU memories. To walk around the limitation, we propose a new method to simulate a high resolution network with a low resolution network, which can be applied during training and/or testing. (ii) Second, we propose an online bootstrapping method for training. We demonstrate that online bootstrapping is critically important for achieving good accuracy. (iii) Third we apply the traditional dropout to some of the residual blocks, which further improves the performance. (iv) Finally, our method achieves the currently best mean intersection-over-union 78.3\% on the PASCAL VOC 2012 dataset, as well as on the recent dataset Cityscapes.

arXiv comment11 pages

References in corpus (4)

• Conditional Random Fields as Recurrent Neural Networks
• Training Very Deep Networks
• Identity Mappings in Deep Residual Networks
• Online Batch Selection for Faster Training of Neural Networks

Cited by in corpus (32)

• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Human pose estimation via Convolutional Part Heatmap Regression
• OCNet: Object Context Network for Scene Parsing
• The Reversible Residual Network: Backpropagation Without Storing Activations
• BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation
• Colorization as a Proxy Task for Visual Understanding
• Global Aggregation then Local Distribution in Fully Convolutional Networks
• Deep Learning based HEp-2 Image Classification: A Comprehensive Review
• Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks
• ACFNet: Attentional Class Feature Network for Semantic Segmentation
• Semantic Segmentation with Labeling Uncertainty and Class Imbalance
• Improving Semantic Segmentation via Self-Training
• Fast LIDAR-based Road Detection Using Fully Convolutional Neural Networks
• Adaptive Affinity Fields for Semantic Segmentation
• Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes
• LMNet: Real-time Multiclass Object Detection on CPU using 3D LiDAR
• LIDAR-based Driving Path Generation Using Fully Convolutional Neural Networks
• Dual ResGCN for Balanced Scene GraphGeneration
• Training Bit Fully Convolutional Network for Fast Semantic Segmentation
• Graph-guided Architecture Search for Real-time Semantic Segmentation
• A Network Structure to Explicitly Reduce Confusion Errors in Semantic Segmentation
• CaseNet: Content-Adaptive Scale Interaction Networks for Scene Parsing
• Diverse Sampling for Self-Supervised Learning of Semantic Segmentation
• Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks
• Dense CNN Learning with Equivalent Mappings
• Attention-based fusion of semantic boundary and non-boundary information to improve semantic segmentation
• Context Prior for Scene Segmentation
• Real-Time Semantic Segmentation via Auto Depth, Downsampling Joint Decision and Feature Aggregation
• Selective Information Passing for MR/CT Image Segmentation
• Learning Rich Representations For Structured Visual Prediction Tasks
• Malleable 2.5D Convolution: Learning Receptive Fields along the Depth-axis for RGB-D Scene Parsing
• High Sensitivity Snapshot Spectrometer Based on Deep Network Unmixing