Improving Semantic Segmentation via Dilated Affinity
arXiv:1907.07011
Abstract
Introducing explicit constraints on the structural predictions has been an effective way to improve the performance of semantic segmentation models. Existing methods are mainly based on insufficient hand-crafted rules that only partially capture the image structure, and some methods can also suffer from the efficiency issue. As a result, most of the state-of-the-art fully convolutional networks did not adopt these techniques. In this work, we propose a simple, fast yet effective method that exploits structural information through direct supervision with minor additional expense. To be specific, our method explicitly requires the network to predict semantic segmentation as well as dilated affinity, which is a sparse version of pair-wise pixel affinity. The capability of telling the relationships between pixels are directly built into the model and enhance the quality of segmentation in two stages. 1) Joint training with dilated affinity can provide robust feature representations and thus lead to finer segmentation results. 2) The extra output of affinity information can be further utilized to refine the original segmentation with a fast propagation process. Consistent improvements are observed on various benchmark datasets when applying our framework to the existing state-of-the-art model. Codes will be released soon.
arXiv comment10 pages, 5 figures, under review of NIPS2019
References in corpus (8)
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
- Conditional Random Fields as Recurrent Neural Networks
- Focal Loss for Dense Object Detection
- ParseNet: Looking Wider to See Better
- The Cityscapes Dataset for Semantic Urban Scene Understanding