Empirical Study of Multi-Task Hourglass Model for Semantic Segmentation Task

Abstract

The semantic segmentation (SS) task aims to create a dense classification by labeling at the pixel level each object present on images. Convolutional neural network (CNN) approaches have been widely used, and exhibited the best results in this task. However, the loss of spatial precision on the results is a main drawback that has not been solved. In this work, we propose to use a multi-task approach by complementing the semantic segmentation task with edge detection, semantic contour, and distance transform tasks. We propose that by sharing a common latent space, the complementary tasks can produce more robust representations that can enhance the semantic labels. We explore the influence of contour-based tasks on latent space, as well as their impact on the final results of SS. We demonstrate the effectiveness of learning in a multi-task setting for hourglass models in the Cityscapes, CamVid, and Freiburg Forest datasets by improving the state-of-the-art without any refinement post-processing.

arXiv commentTo appear in IEEE Access. Code available at https://gitlab.com/mipl/mtl-ss

References in corpus (16)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Conditional Random Fields as Recurrent Neural Networks
• An Overview of Multi-Task Learning in Deep Neural Networks
• ParseNet: Looking Wider to See Better
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• A Survey on Multi-Task Learning
• Hierarchical Multi-Scale Attention for Semantic Segmentation
• Self-Supervised Model Adaptation for Multimodal Semantic Segmentation
• FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation
• Learning Multiple Tasks with Multilinear Relationship Networks
• A Survey of Semantic Segmentation
• Learning to Branch for Multi-Task Learning
• Semantic Edge Detection with Diverse Deep Supervision
• Efficient Smoothing of Dilated Convolutions for Image Segmentation
• Multi-Task Learning for Automotive Foggy Scene Understanding via Domain Adaptation to an Illumination-Invariant Representation

Cited by in corpus (1)

• Global and Local Features through Gaussian Mixture Models on Image Semantic Segmentation