Beyond Forward Shortcuts: Fully Convolutional Master-Slave Networks (MSNets) with Backward Skip Connections for Semantic Segmentation

Abstract

Recent deep CNNs contain forward shortcut connections; i.e. skip connections from low to high layers. Reusing features from lower layers that have higher resolution (location information) benefit higher layers to recover lost details and mitigate information degradation. However, during inference the lower layers do not know about high layer features, although they contain contextual high semantics that benefit low layers to adaptively extract informative features for later layers. In this paper, we study the influence of backward skip connections which are in the opposite direction to forward shortcuts, i.e. paths from high layers to low layers. To achieve this -- which indeed runs counter to the nature of feed-forward networks -- we propose a new fully convolutional model that consists of a pair of networks. A `Slave' network is dedicated to provide the backward connections from its top layers to the `Master' network's bottom layers. The Master network is used to produce the final label predictions. In our experiments we validate the proposed FCN model on ADE20K (ImageNet scene parsing), PASCAL-Context, and PASCAL VOC 2011 datasets.

9 pages, 5 figures

References in corpus (18)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Residual Learning for Image Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Densely Connected Convolutional Networks
• Wide Residual Networks
• ParseNet: Looking Wider to See Better
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Resnet in Resnet: Generalizing Residual Architectures
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Pyramid Scene Parsing Network
• Deep Networks with Stochastic Depth
• Semantic Understanding of Scenes through the ADE20K Dataset
• Bridging Category-level and Instance-level Semantic Image Segmentation
• Swapout: Learning an ensemble of deep architectures
• The Importance of Skip Connections in Biomedical Image Segmentation
• The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
• PixelNet: Towards a General Pixel-level Architecture
• Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation

Cited by in corpus (3)

• Fully Convolutional Networks for Chip-wise Defect Detection Employing Photoluminescence Images
• Partial Labeled Gastric Tumor Segmentation via patch-based Reiterative Learning
• Beyond Single Stage Encoder-Decoder Networks: Deep Decoders for Semantic Image Segmentation