RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation

Abstract

Recently, very deep convolutional neural networks (CNNs) have shown outstanding performance in object recognition and have also been the first choice for dense classification problems such as semantic segmentation. However, repeated subsampling operations like pooling or convolution striding in deep CNNs lead to a significant decrease in the initial image resolution. Here, we present RefineNet, a generic multi-path refinement network that explicitly exploits all the information available along the down-sampling process to enable high-resolution prediction using long-range residual connections. In this way, the deeper layers that capture high-level semantic features can be directly refined using fine-grained features from earlier convolutions. The individual components of RefineNet employ residual connections following the identity mapping mindset, which allows for effective end-to-end training. Further, we introduce chained residual pooling, which captures rich background context in an efficient manner. We carry out comprehensive experiments and set new state-of-the-art results on seven public datasets. In particular, we achieve an intersection-over-union score of 83.4 on the challenging PASCAL VOC 2012 dataset, which is the best reported result to date.

References in corpus (16)

• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Multi-Scale Context Aggregation by Dilated Convolutions
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• MatConvNet - Convolutional Neural Networks for MATLAB
• Learning Deconvolution Network for Semantic Segmentation
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Identity Mappings in Deep Residual Networks
• Semantic Understanding of Scenes through the ADE20K Dataset
• Semantic Image Segmentation via Deep Parsing Network
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Attention to Scale: Scale-aware Semantic Image Segmentation
• Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts
• Semantic Object Parsing with Local-Global Long Short-Term Memory
• Zoom Better to See Clearer: Human and Object Parsing with Hierarchical Auto-Zoom Net
• Semantic Object Parsing with Graph LSTM

Cited by in corpus (19)

• Diffusion Models Beat GANs on Image Synthesis
• Learning Depth from Single Images with Deep Neural Network Embedding Focal Length
• PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing
• FermiNets: Learning generative machines to generate efficient neural networks via generative synthesis
• Real-time Semantic Image Segmentation via Spatial Sparsity
• Fully Convolutional Networks for Chip-wise Defect Detection Employing Photoluminescence Images
• Lucid Data Dreaming for Video Object Segmentation
• Adapting Semantic Segmentation Models for Changes in Illumination and Camera Perspective
• DARTS: DenseUnet-based Automatic Rapid Tool for brain Segmentation
• Deep attention-based classification network for robust depth prediction
• SAN: Scale-Aware Network for Semantic Segmentation of High-Resolution Aerial Images
• DeepSkeleton: Skeleton Map for 3D Human Pose Regression
• A Network Structure to Explicitly Reduce Confusion Errors in Semantic Segmentation
• Learning with Hierarchical Complement Objective
• Rethinking Convolutional Semantic Segmentation Learning
• Sky Optimization: Semantically aware image processing of skies in low-light photography
• Semantic Segmentation via Highly Fused Convolutional Network with Multiple Soft Cost Functions
• Issues in Object Detection in Videos using Common Single-Image CNNs
• Rethinking Fully Convolutional Networks for the Analysis of Photoluminescence Wafer Images